-- Logs begin at Fri 2022-09-09 05:49:15 UTC, end at Fri 2022-09-09 06:48:24 UTC. -- Sep 09 06:14:37.322438 dv-cinder-852209-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 06:14:37.331134 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 06:14:37 2022] *** Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: nodename: dv-cinder-852209-16-iscsi Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: machine: x86_64 Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: clock source: unix Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: pcre jit disabled Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: detected number of CPU cores: 4 Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: current working directory: / Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: detected binary path: /usr/bin/uwsgi-core Sep 09 06:14:37.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: your processes number limit is 47680 Sep 09 06:14:37.334215 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: your memory page size is 4096 bytes Sep 09 06:14:37.334215 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: detected max file descriptor number: 2048 Sep 09 06:14:37.334215 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: lock engine: pthread robust mutexes Sep 09 06:14:37.334215 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: thunder lock: enabled Sep 09 06:14:37.334215 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 06:14:37.334215 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 06:14:37.373773 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: Python main interpreter initialized at 0x5571f67e29c0 Sep 09 06:14:37.373773 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: python threads support enabled Sep 09 06:14:37.373773 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: your server socket listen backlog is limited to 100 connections Sep 09 06:14:37.373773 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: your mercy for graceful operations on workers is 90 seconds Sep 09 06:14:37.374311 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 06:14:37.374311 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: *** Operational MODE: preforking *** Sep 09 06:14:37.374594 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: *** uWSGI is running in multiple interpreter mode *** Sep 09 06:14:37.374594 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: spawned uWSGI master process (pid: 81467) Sep 09 06:14:37.374807 dv-cinder-852209-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 06:14:37.376416 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: spawned uWSGI worker 1 (pid: 81480, cores: 1) Sep 09 06:14:37.376416 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: spawned uWSGI worker 2 (pid: 81481, cores: 1) Sep 09 06:14:37.376416 dv-cinder-852209-16-iscsi devstack@keystone.service[81467]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 06:14:41.751498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 06:14:41.751498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Sep 09 06:14:41.752319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 06:14:41.752319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Sep 09 06:14:41.752457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 06:14:41.752457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: from cryptography.utils import int_from_bytes Sep 09 06:14:41.752457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 06:14:41.752457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: from cryptography.utils import int_from_bytes Sep 09 06:14:41.802082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 06:14:41.802259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 06:14:41.802377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] command line args: [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 06:14:41.802599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 06:14:41.802683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 06:14:41.802792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 06:14:41.802884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ================================================================================ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 06:14:41.802988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] command line args: [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 06:14:41.803073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.803258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 06:14:41.803343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_dir = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.803425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ================================================================================ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 06:14:41.803496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.803624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_source = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.803709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] admin_token = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.803803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] debug = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.803950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_dir = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_source = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] debug = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.804910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_config_append = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_dir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.805927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_config_append = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_dir = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.806920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807742 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.807947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.808882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809032 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] publish_errors = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.809958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] publish_errors = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.810884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.811000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.811821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] standard_threads = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] standard_threads = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.812939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_journal = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_json = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_stderr = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_journal = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_syslog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.813933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_json = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.814052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.814165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_stderr = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.814165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_syslog = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.814382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.814494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 06:14:41.814605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.814731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.814845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.814963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.815913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.816957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.817993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.818982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.819980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820096 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.820886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.821925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.822953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.823900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.824975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.825982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.826717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.827393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.827590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.827755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.827970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.828900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.829883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.830924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.831038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.831151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.831263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.831380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.831501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.831917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.832951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833528 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.833995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.834919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.835973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.836953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.837999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.838901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.839976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.external = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.840943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.external = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.password = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.token = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.841963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.842084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.password = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.842198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.842338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.token = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.842574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.842740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.842927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.843920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.844962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.845990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.846910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.847965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.848905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.849946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.850954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.851944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.852996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.853905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.854925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.855046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.855161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.855275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.855662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.855781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.855894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.856950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.857875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.858952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859304 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.859958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.860870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.861894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.862933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.863998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.864916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.865956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.866071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.866185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.866298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.866753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.866869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.866992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.867943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.868978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.869907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.870948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.871910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.872945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.873058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.873180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.873296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.873409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.873527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.874507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.874626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.874862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.874994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.875961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.876899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.877949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.878984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.879950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.880964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.881991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.882907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.883894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.884929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.885967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.886904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.887986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.888907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.889021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.889144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.889898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.890965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.891909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 06:14:41.892776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 06:14:41.892776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 06:14:41.917822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.918048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.918271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.918566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.918798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.919022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.927404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.927897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.956557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.956557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.958301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:41.958531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.011109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.011713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.059878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.060056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.060145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.060292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.217878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.218005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.218086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.218178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.218257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.218339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.241397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.241645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.241742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.241856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.241936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242293 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.242943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.253874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.254386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.267192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.267419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.267596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.267712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.267849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.267938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.268873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 06:14:42.285206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.285408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.285493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 06:14:42.285617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 06:14:42.285748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.285950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 06:14:42.286032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.286145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 06:14:42.286229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.286415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.286505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.286679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.286759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.286938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.287018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.287195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.296771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.296873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.297045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.297128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.297313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.297410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.297594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.297676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.304527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.304623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.307402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.307486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.307709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.307793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.307992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.308909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.309937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.310023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.310165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.310245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.333778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.334168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.335238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.335523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.335714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.335999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.340216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.340503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.340647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.340788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.340931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.341211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.347704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.347993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.348138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.348422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.353614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.353916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.353998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.354205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.354293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.354496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.354580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.354779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.354862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.355900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.371257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.371402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.371536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.371705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.376149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.376338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.379379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.379569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.379706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.379873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.383915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.384147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.387157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.387336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.390496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.390733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.390820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.391980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.392182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.408450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.408553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.408740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.408824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.409005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.409101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.409265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.409348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.416086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.416181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.417616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.417749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.419142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.419225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.419418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.419500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.422193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.422281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.422475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.422561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.422756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.422849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.423037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.423120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.423433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.423518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.434492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.434582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.434780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.434864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.435063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.435160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.435346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.435430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.435752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.435838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.436548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.436639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.436847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.437022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.452349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.452679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.455259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.455579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.458472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.458725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.459919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.460280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.461390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.461672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.461770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.462011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.462152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.462401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.469097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.469394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.470560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.470822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.473454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.473711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.473855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.474105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.474187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.474390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.474482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.474668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.482221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.482422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.482517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.482710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.482793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.483052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.489696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.489931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 06:14:42.490025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.490214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.490306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.490493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.490586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.490770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.490861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.491986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.492967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.493054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.493227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.493311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.493502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 06:14:42.527564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5571f67e29c0 pid: 81480 (default app) Sep 09 06:14:42.527828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5571f67e29c0 pid: 81481 (default app) Sep 09 06:14:42.531439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71134b-4d73-4fc1-8e40-3fdcd31d4c08 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:42.531743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71134b-4d73-4fc1-8e40-3fdcd31d4c08 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:42.532015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71134b-4d73-4fc1-8e40-3fdcd31d4c08 None None] PATH_INFO: `/v3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:42.536144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Fri Sep 9 06:14:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:14:49.326722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:49.328291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:49.328291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:49.332208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:14:49.337495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:49.337975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:49.338067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:49.409052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: INFO dbcounter [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] Registered counter for database keystone Sep 09 06:14:49.427928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG oslo_db.sqlalchemy.engines [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81480) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 06:14:49.434056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writer thread running {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 06:14:49.613696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81480) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 06:14:49.614012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81480) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 06:14:49.671929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:49.673539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:49.801048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 06:14:50.225172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:50.294334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: INFO dbcounter [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] Registered counter for database keystone Sep 09 06:14:50.315715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG oslo_db.sqlalchemy.engines [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81481) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 06:14:50.321798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writer thread running {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 06:14:50.442601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abba2f9-7059-4605-b7ed-25f749c28e96 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:50.446847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:50.447476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:50.447790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:50.448064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:50.457154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:50.508267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING keystone.common.rbac_enforcer.enforcer [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 06:14:50.508558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c80984-17d5-45c6-98d6-5e2bec3011d5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:50.518334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Fri Sep 9 06:14:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:14:50.524450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:50.541911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f3d35-36d0-4f7b-9870-f4d1696d2701 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:50.546981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:50.547655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:50.547946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:50.548222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] PATH_INFO: `/v3/projects/df82cd1e5393473c92978bf22c792e3e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:50.549810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] RBAC: Authorizing `identity:get_project(project_id=df82cd1e5393473c92978bf22c792e3e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:50.602758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 06:14:50.603105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882f616-7e4d-406e-9d6f-77975bd35d7a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:50.607643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Fri Sep 9 06:14:50 2022] GET /identity/v3/projects/df82cd1e5393473c92978bf22c792e3e => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:14:52.737798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117aea73-94d4-4ff3-91d3-9a8c7637be50 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:52.738308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117aea73-94d4-4ff3-91d3-9a8c7637be50 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:52.738415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117aea73-94d4-4ff3-91d3-9a8c7637be50 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:52.742456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:14:52.748768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:52.749194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:52.749478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:52.769207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:52.770719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:52.778479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 06:14:53.048729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117aea73-94d4-4ff3-91d3-9a8c7637be50 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:53.060666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117aea73-94d4-4ff3-91d3-9a8c7637be50 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:53.064855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366d743-291f-4e5e-987e-91617a7d836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:53.065444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366d743-291f-4e5e-987e-91617a7d836e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:53.065718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366d743-291f-4e5e-987e-91617a7d836e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:53.065984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366d743-291f-4e5e-987e-91617a7d836e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:53.073662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366d743-291f-4e5e-987e-91617a7d836e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:53.075420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366d743-291f-4e5e-987e-91617a7d836e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:53.084343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Fri Sep 9 06:14:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:14:53.089756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:53.099325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fcda2-eec3-4c9c-87d5-1776f3a5edf9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:53.103826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52e748-d39c-457d-8e7c-fc4a4f20771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:53.104459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e748-d39c-457d-8e7c-fc4a4f20771d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:53.104741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e748-d39c-457d-8e7c-fc4a4f20771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:53.105014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52e748-d39c-457d-8e7c-fc4a4f20771d admin admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:53.106335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52e748-d39c-457d-8e7c-fc4a4f20771d admin admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:53.108769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52e748-d39c-457d-8e7c-fc4a4f20771d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:53.110293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Fri Sep 9 06:14:53 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:14:55.319725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f75c4-e354-439c-8e25-909363c2391f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:55.320282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f75c4-e354-439c-8e25-909363c2391f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:55.320282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f75c4-e354-439c-8e25-909363c2391f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:55.321577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:14:55.327988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:55.328507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:55.328945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:55.346751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:55.347946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:55.408020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb094c-3d63-4cdf-a4bb-77c6f2f9449c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:55.408303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb094c-3d63-4cdf-a4bb-77c6f2f9449c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:55.408582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb094c-3d63-4cdf-a4bb-77c6f2f9449c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:55.409907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:14:55.414772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:55.415051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:55.415324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:55.430199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:14:55.458377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG passlib.handlers.bcrypt [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81481) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 06:14:55.458774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG passlib.handlers.bcrypt [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81481) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 06:14:55.508082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:55.509647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:55.608477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:14:55.708125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:55.731907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:55.815080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ac92e-aa49-4724-a4f7-ab86a4fd1c80 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:55.848400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:55.849038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:55.849320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:55.849599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:55.861225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:14:55.870067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:14:55.875604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:14:55.876541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:14:55.877238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:55.877310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:55.959285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:55.978551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06a9d9-422a-4bfe-a3aa-115c1a7a2286 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:55.978727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59c307-0305-4357-b5af-b5edab9af42f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:55.980939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:56.013499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.014109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.014392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.014668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.015372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:14:56.015702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.017088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.019977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.020846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.021127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.021399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.028745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:14:56.035377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:56.091958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.093477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.136199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1f13a-caf4-46a4-8828-bd6b920d4506 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:56.170894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.171536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.171846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.172125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.173269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.174732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.176326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 9/16] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:14:56 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:14:56.181709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:56.189363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 8/17] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:14:56.194703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:56.196494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:56.197360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:56.282284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639eb5a-7f70-42a3-aeed-82a7f3de993d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:56.297815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88292f6-7ce7-4b55-9b51-4526b594db57 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:56.317753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.318388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.318668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.318941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.320135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.321584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.323147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:14:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:14:56.328992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:56.336303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.337499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.337499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.337698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.356274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.358577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:14:56.377173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:14:56.378081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:14:56.379117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:14:56.379117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:14:56.379117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:14:56.379117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:14:56.383974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:56.410999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:56.430514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601e99-46ee-4b5b-adb1-e758062130e2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:56.439958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:56.464636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.465244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.465525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.465800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.466587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.domain.id=default, user.id=285daab328104f75886dde4ae3928fec) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:14:56.466900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.468358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf215d-15fe-4c47-9046-34727cb66af1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.474856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Fri Sep 9 06:14:56 2022] GET /identity/v3/role_assignments?user.id=285daab328104f75886dde4ae3928fec&scope.domain.id=default&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:14:56.486383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c584c0a-c350-4c41-a3af-c145a0628919 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:56.522650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:56.523277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:56.523596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:56.523860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:56.524527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:14:56.524831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:14:56.526271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cb28c-6d0a-4b9d-9375-4e9a57d63c4f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:14:56.550104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:14:58.860666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab2b48-4ef2-454b-be94-50657e21b389 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:58.861663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab2b48-4ef2-454b-be94-50657e21b389 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:58.862166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab2b48-4ef2-454b-be94-50657e21b389 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:58.863746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:14:58.875752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee7f38-c0c3-4f6a-a6f4-83b52ba80406 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:58.876610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee7f38-c0c3-4f6a-a6f4-83b52ba80406 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:58.877093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee7f38-c0c3-4f6a-a6f4-83b52ba80406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:58.905609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c6ee7f38-c0c3-4f6a-a6f4-83b52ba80406 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:58.907198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c6ee7f38-c0c3-4f6a-a6f4-83b52ba80406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:58.992435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 11/23] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:14:59.529970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab2b48-4ef2-454b-be94-50657e21b389 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:59.607140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59a4e5-64cb-4c09-86a3-706373939cfe None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.607810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59a4e5-64cb-4c09-86a3-706373939cfe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.608290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59a4e5-64cb-4c09-86a3-706373939cfe None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.609861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:14:59.617375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb753d36-ef18-4488-8a44-2c2ce8abb694 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.617938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb753d36-ef18-4488-8a44-2c2ce8abb694 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.618421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb753d36-ef18-4488-8a44-2c2ce8abb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.643624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fb753d36-ef18-4488-8a44-2c2ce8abb694 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:59.645969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb753d36-ef18-4488-8a44-2c2ce8abb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:59.726813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab2b48-4ef2-454b-be94-50657e21b389 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:14:59.731570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:59.783646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 13/25] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:14:59.785437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a73fd-b2f7-41a8-b53a-70761e7fb51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:14:59.786333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a73fd-b2f7-41a8-b53a-70761e7fb51f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.786817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a73fd-b2f7-41a8-b53a-70761e7fb51f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.787301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a73fd-b2f7-41a8-b53a-70761e7fb51f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.789159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1ef20-b70b-49e8-bb69-7a5022ce09a0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.789632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1ef20-b70b-49e8-bb69-7a5022ce09a0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.790067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1ef20-b70b-49e8-bb69-7a5022ce09a0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.791518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 14/26] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:14:59.796981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918112b-367e-4c78-8dbe-2898fdce1bd0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.797445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918112b-367e-4c78-8dbe-2898fdce1bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.797890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918112b-367e-4c78-8dbe-2898fdce1bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.799568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a73fd-b2f7-41a8-b53a-70761e7fb51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-be6a73fd-b2f7-41a8-b53a-70761e7fb51f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:14:59.805791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:14:59.806635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:14:59.807418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:14:59.809898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 13/27] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:14:59.811875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b36ad-6f53-4f66-a348-0f47129aaea3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.812397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b36ad-6f53-4f66-a348-0f47129aaea3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.812878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b36ad-6f53-4f66-a348-0f47129aaea3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.814570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 14/28] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:14:59.816004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b36ad-6f53-4f66-a348-0f47129aaea3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:14:59.817687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6918112b-367e-4c78-8dbe-2898fdce1bd0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:59.818955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6918112b-367e-4c78-8dbe-2898fdce1bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:14:59.831903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:14:59.890908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 15/29] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:14:59.892824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:14:59.893315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:14:59.893813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:14:59.912021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:14:59.913522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:00.000666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 16/30] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:00.013898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b36ad-6f53-4f66-a348-0f47129aaea3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.088937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.089487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.089775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.090057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.090764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:00.091051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.092632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.108297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 15/31] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:00.114138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.146374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.160001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:00.232788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:00.271291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1fa8-1c39-447d-ae83-9d811dc58397 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.311994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c89eb-0d69-4cf8-b8e2-d2bac0e49c6b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.323567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.323567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.323567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.323567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.323928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.325154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.372382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.373505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.373809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.374084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.376306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.377768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.379425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 17/32] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:00 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:00.384879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.487374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 16/33] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Sep 9 06:15:00 2022] POST /identity/v3/roles => generated 219 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:15:00.491777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.529630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f13722-abe8-4016-8d9a-bb2f21460917 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.534972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:00.608628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.609292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.609567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.609834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.611270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.612756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.621638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 18/34] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:00.623233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.687247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b873a7-73d1-4375-8ea9-d25773d52299 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.725641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:00.742580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.743629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.744129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.744955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.760797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.762423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.777112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6addd-52f4-4bfb-8ff4-a7ed7759de21 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:15:00.783014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:00.784326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:15:00.785134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:15:00.785134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:15:00.785134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:00.785134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 17/35] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:00.785134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.828835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ce103-12c0-440c-a36a-6bafe4461678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.829458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce103-12c0-440c-a36a-6bafe4461678 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.829741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce103-12c0-440c-a36a-6bafe4461678 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.830015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ce103-12c0-440c-a36a-6bafe4461678 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.830696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ce103-12c0-440c-a36a-6bafe4461678 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.832405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ce103-12c0-440c-a36a-6bafe4461678 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.900622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbd412-267f-44d4-9428-6c1d2f1bd1f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:00.934139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce410831-7202-47ea-8348-937d3ed1c9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:00.934803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce410831-7202-47ea-8348-937d3ed1c9aa None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.935093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce410831-7202-47ea-8348-937d3ed1c9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.935377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce410831-7202-47ea-8348-937d3ed1c9aa None admin] PATH_INFO: `/v3/domains/default/users/285daab328104f75886dde4ae3928fec/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.937462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce410831-7202-47ea-8348-937d3ed1c9aa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=285daab328104f75886dde4ae3928fec, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:00.938910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce410831-7202-47ea-8348-937d3ed1c9aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:00.953738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 18/36] 10.222.0.25 () {58 vars in 1361 bytes} [Fri Sep 9 06:15:00 2022] PUT /identity/v3/domains/default/users/285daab328104f75886dde4ae3928fec/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:00.955059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8be5e-d681-43d1-b090-f94bf498940b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.955350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8be5e-d681-43d1-b090-f94bf498940b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.955653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8be5e-d681-43d1-b090-f94bf498940b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.957191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 19/37] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:00.957961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8be5e-d681-43d1-b090-f94bf498940b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:00.967359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 19/38] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Sep 9 06:15:00 2022] POST /identity/v3/roles => generated 225 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:00.969129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede792-dd07-424a-8fce-76185b990908 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.969423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede792-dd07-424a-8fce-76185b990908 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.969705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede792-dd07-424a-8fce-76185b990908 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.971391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 20/39] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:00.972478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84478a-dab5-41da-b421-407492c20fc4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:00.972820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84478a-dab5-41da-b421-407492c20fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:00.973103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84478a-dab5-41da-b421-407492c20fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:00.994831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ac84478a-dab5-41da-b421-407492c20fc4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:00.996541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ac84478a-dab5-41da-b421-407492c20fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:01.031214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:01.063305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:01.095880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 21/40] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:01.097680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:01.098208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:01.098680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:01.121948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:01.123804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:01.199302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8be5e-d681-43d1-b090-f94bf498940b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:01.222709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 22/41] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:01.254370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:01.254999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:01.255278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:01.255567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:01.256243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:01.256547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:01.257914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:01.279816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 20/42] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:01.284308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:01.383014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:01.393062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534feab8-8012-4804-b59f-9b3265d1d188 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:01.426192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:01.429878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:01.431463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:01.431778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:01.432055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:01.432720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:01.434299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:01.472035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 23/43] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:15:01 2022] POST /identity/v3/projects => generated 301 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:01.525357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:01.571438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:01.577667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90f067-f451-4fc6-b244-f830c91d00b1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:01.627240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:01.627936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:01.628224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:01.628502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:01.630218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:01.632594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:01.634586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 21/44] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:01.640945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:01.670629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e552b-445f-43c0-8aac-dc5b35c50a37 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:01.714217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2d390-e79c-4e78-9dcf-d51251236915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:01.715017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2d390-e79c-4e78-9dcf-d51251236915 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:01.715453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2d390-e79c-4e78-9dcf-d51251236915 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:01.715865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2d390-e79c-4e78-9dcf-d51251236915 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:01.717448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2d390-e79c-4e78-9dcf-d51251236915 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:01.719177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2d390-e79c-4e78-9dcf-d51251236915 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:01.736783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 24/45] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Sep 9 06:15:01 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:15:01.877482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26ebb0-7bd1-48f6-bbd7-61bfb793c301 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:01.919167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07558f-8094-410b-827b-c3d1629f450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:01.920420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07558f-8094-410b-827b-c3d1629f450d None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:01.920709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07558f-8094-410b-827b-c3d1629f450d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:01.920989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07558f-8094-410b-827b-c3d1629f450d None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:01.921670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07558f-8094-410b-827b-c3d1629f450d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:01.923117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07558f-8094-410b-827b-c3d1629f450d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:01.958233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 22/46] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:15:01 2022] POST /identity/v3/projects => generated 312 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:03.239609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813ae6-09ae-42dc-b9f7-752256e34575 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:03.240124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813ae6-09ae-42dc-b9f7-752256e34575 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:03.240238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813ae6-09ae-42dc-b9f7-752256e34575 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:03.241493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 25/47] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:03.246797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:03.247179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:03.247985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:03.266421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:03.269149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:03.282996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:03.346059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 23/48] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:03.615815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813ae6-09ae-42dc-b9f7-752256e34575 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:03.619710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:03.720413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813ae6-09ae-42dc-b9f7-752256e34575 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:03.754242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:03.754868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:03.755149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:03.755420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:03.767118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:03.773144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:03.774100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:03.774858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 26/49] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:03.780392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:03.878445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb730259-39d6-4a87-b059-a890c2f1d06f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:03.912148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:03.912764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:03.913045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:03.913317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:03.913986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:03.914292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:03.915693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:03.924189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 24/50] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:03.929917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:03.992332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:04.031926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27bf05-0b4e-43d8-8eb9-b59cd1924cf4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:04.067788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2be11-1664-4635-9704-9eb3ac3c4aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:04.068424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2be11-1664-4635-9704-9eb3ac3c4aec None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.068702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2be11-1664-4635-9704-9eb3ac3c4aec None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.068974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2be11-1664-4635-9704-9eb3ac3c4aec None admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.070098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2be11-1664-4635-9704-9eb3ac3c4aec None admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:04.071559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2be11-1664-4635-9704-9eb3ac3c4aec None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:04.073117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 27/51] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:03 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:04.078717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:04.091942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:04.148107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f70c84-14f5-4b93-b901-ce0d46908164 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.148402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f70c84-14f5-4b93-b901-ce0d46908164 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.148667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f70c84-14f5-4b93-b901-ce0d46908164 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.150136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 28/52] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:04.155538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.155838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.156113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.174305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:04.175487 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:04.182221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f75c5-0778-479a-9b38-d0e0ba650b8b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:04.216903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:04.217537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.217858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.218138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.219254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:04.220744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:04.222317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 25/53] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:04.228167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:04.242445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 29/54] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:04.305460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:04.326367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e0425-ae8e-4ae9-9416-e99caf74482a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:04.360290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:04.360899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.361183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.361461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.362192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.domain.id=default, user.id=285daab328104f75886dde4ae3928fec) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:04.362506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:04.363998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:04.369736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 26/55] 10.222.0.25 () {56 vars in 1322 bytes} [Fri Sep 9 06:15:04 2022] GET /identity/v3/role_assignments?user.id=285daab328104f75886dde4ae3928fec&scope.domain.id=default&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:04.514882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:04.581664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:04.624394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4e2db-2697-4519-a1d6-cb8955613940 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:04.759866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5754d10-ff82-4980-ba4a-bb65e8e7a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:04.760495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5754d10-ff82-4980-ba4a-bb65e8e7a87a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.760778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5754d10-ff82-4980-ba4a-bb65e8e7a87a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.761058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5754d10-ff82-4980-ba4a-bb65e8e7a87a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.762162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5754d10-ff82-4980-ba4a-bb65e8e7a87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:04.764325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5754d10-ff82-4980-ba4a-bb65e8e7a87a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:04.765930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 30/56] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:04.771664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:04.869346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a3224-985c-4505-a8a3-0e645db8445f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:04.902363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8a8ce-be5d-4a3e-ab80-e1dfd6a03694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:04.902969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8a8ce-be5d-4a3e-ab80-e1dfd6a03694 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:04.903252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8a8ce-be5d-4a3e-ab80-e1dfd6a03694 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:04.903614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8a8ce-be5d-4a3e-ab80-e1dfd6a03694 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:04.904423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8a8ce-be5d-4a3e-ab80-e1dfd6a03694 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:04.905865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8a8ce-be5d-4a3e-ab80-e1dfd6a03694 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:04.920224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:04.931178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 27/57] 10.222.0.25 () {60 vars in 1140 bytes} [Fri Sep 9 06:15:04 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:07.068613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ea5a0-cb0c-41af-a4fb-3701908c9606 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:07.069126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ea5a0-cb0c-41af-a4fb-3701908c9606 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:07.069210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ea5a0-cb0c-41af-a4fb-3701908c9606 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:07.070495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 31/58] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:07.076471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:07.076862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:07.077038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:07.094949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:07.096175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:07.161245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 28/59] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:07.431240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ea5a0-cb0c-41af-a4fb-3701908c9606 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:07.528762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ea5a0-cb0c-41af-a4fb-3701908c9606 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:07.555822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:07.562490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f662a-15fa-4a7c-a216-c0c6959a2810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:07.563118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f662a-15fa-4a7c-a216-c0c6959a2810 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:07.563397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f662a-15fa-4a7c-a216-c0c6959a2810 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:07.563695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f662a-15fa-4a7c-a216-c0c6959a2810 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:07.564820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f662a-15fa-4a7c-a216-c0c6959a2810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:07.566255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f662a-15fa-4a7c-a216-c0c6959a2810 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:07.567846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 32/60] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:07.573245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:07.668865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa07672-240d-4b81-8403-4f01407da7e4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:07.691288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:07.703006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea078dd-f869-4a2a-8321-b4ff2bce22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:07.703639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea078dd-f869-4a2a-8321-b4ff2bce22a1 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:07.703926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea078dd-f869-4a2a-8321-b4ff2bce22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:07.704202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea078dd-f869-4a2a-8321-b4ff2bce22a1 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:07.704864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea078dd-f869-4a2a-8321-b4ff2bce22a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:07.706561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea078dd-f869-4a2a-8321-b4ff2bce22a1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:07.733179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 29/61] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:07 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:15:10.950421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe861ccb-6f2e-4c52-9b60-247dfbb26d99 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:10.951363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe861ccb-6f2e-4c52-9b60-247dfbb26d99 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:10.951849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe861ccb-6f2e-4c52-9b60-247dfbb26d99 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:10.953402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 33/62] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:10.959322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31338cf-3f50-4844-824f-bec87d092ba9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:10.960125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31338cf-3f50-4844-824f-bec87d092ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:10.960603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31338cf-3f50-4844-824f-bec87d092ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:10.988566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e31338cf-3f50-4844-824f-bec87d092ba9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:10.990068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e31338cf-3f50-4844-824f-bec87d092ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:11.081138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 30/63] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:11.220628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fcdca-26fa-4801-b3d3-454227172e6b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.220807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fcdca-26fa-4801-b3d3-454227172e6b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.221114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fcdca-26fa-4801-b3d3-454227172e6b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.222454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 34/64] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:11.241259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58dad0-2a6e-4227-9e44-bbcda5b45605 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.241807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58dad0-2a6e-4227-9e44-bbcda5b45605 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.242267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58dad0-2a6e-4227-9e44-bbcda5b45605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.268721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9d58dad0-2a6e-4227-9e44-bbcda5b45605 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:11.270279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d58dad0-2a6e-4227-9e44-bbcda5b45605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:11.361366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 31/65] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:11.393310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8c30-0dd8-48a9-b6f3-bcdfb6c4cd06 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.393934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8c30-0dd8-48a9-b6f3-bcdfb6c4cd06 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.394433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e8c30-0dd8-48a9-b6f3-bcdfb6c4cd06 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.395999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 35/66] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:11.401661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562a72a-37bf-4031-9388-51e40836e755 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.402177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562a72a-37bf-4031-9388-51e40836e755 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.402644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562a72a-37bf-4031-9388-51e40836e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.431858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e562a72a-37bf-4031-9388-51e40836e755 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:11.433370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e562a72a-37bf-4031-9388-51e40836e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:11.449796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8c30-0dd8-48a9-b6f3-bcdfb6c4cd06 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:11.500125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:11.579126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 32/67] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:11.605157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e8c30-0dd8-48a9-b6f3-bcdfb6c4cd06 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:11.655496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:11.656565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.657052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.657553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.664704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a3c91-1336-41d0-8a17-dfa2107b4841 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.665316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a3c91-1336-41d0-8a17-dfa2107b4841 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.665781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a3c91-1336-41d0-8a17-dfa2107b4841 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.667335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 33/68] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:11.679963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:11.680927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.681408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.681872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:11.685482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:11.686330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:11.687107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 36/69] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:11.691038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:11.711262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:11.713006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:11.756012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:11.790400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 34/70] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:11.791595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:11.806084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bd09c-153d-4bca-9f7d-85bcef5aa12a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:11.843103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430dc44-349b-4882-a28b-adbcbf668694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:11.844092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430dc44-349b-4882-a28b-adbcbf668694 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.844574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430dc44-349b-4882-a28b-adbcbf668694 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.845100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430dc44-349b-4882-a28b-adbcbf668694 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.855844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430dc44-349b-4882-a28b-adbcbf668694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e430dc44-349b-4882-a28b-adbcbf668694 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:11.861584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:11.862419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:11.863194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:11.865813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 37/71] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:11.867617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db358c-497c-4386-9dd7-118f650da17b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.868125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db358c-497c-4386-9dd7-118f650da17b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.868593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db358c-497c-4386-9dd7-118f650da17b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.870125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 38/72] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:11.876274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db358c-497c-4386-9dd7-118f650da17b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:11.894294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:11.898799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba64e1-ceee-42e6-bcb6-54ff530eeacb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:11.934952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:11.935598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.935883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.936162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.936836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:11.937143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:11.938512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e17bcc-4fca-44d6-b9e3-18fcbf421209 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:11.952358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 35/73] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:11.952358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c154194-011e-4817-8219-df355740a1e1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:11.952358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c154194-011e-4817-8219-df355740a1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:11.952358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c154194-011e-4817-8219-df355740a1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:11.968431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5c154194-011e-4817-8219-df355740a1e1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:11.969652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c154194-011e-4817-8219-df355740a1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:11.992243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db358c-497c-4386-9dd7-118f650da17b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.010694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.030597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.031211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.031500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.031790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.032674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:12.032799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.034109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.040225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 36/74] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:12.040880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c154194-011e-4817-8219-df355740a1e1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.062627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 39/75] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.067454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.138805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.148946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c154194-011e-4817-8219-df355740a1e1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.182691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aba9a-2b63-43f5-b9c2-0166a0db9f10 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.186058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.186793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.187081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.187356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.197946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:12.203171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:12.204062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:12.204831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 37/76] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:12.205484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.224733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.225659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.226115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.226600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.243476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.244977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.246618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 40/77] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.248388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.289887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.314178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337beaa-18d7-4a11-9806-2b8782dd30ad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.350464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.351103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.351388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.351760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.353082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eadde4-e2ac-44d9-9961-d08adac71442 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.362790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:12.367738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:12.368705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:12.369495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:12.373623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 38/78] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:12.374411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.387204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.387990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.388271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.388545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.389297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.390008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.391939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.393641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 41/79] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.394266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.475835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0479a-51b8-4e6b-9988-c21bde5da2da None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.496112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2011d18-ae73-4345-a9b0-b0af90ab504b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.503166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.509990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.510600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.510880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.511154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.511854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:12.512162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.513579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.521461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 39/80] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.522257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.531320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.531968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.532248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.532519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.598371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.601216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.602930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.625127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2301a2-d5f9-4c27-b63d-4755fad16552 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.658420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.659055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.659337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.659651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.660764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.662199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.663787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 40/81] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.664554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.702386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 42/82] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:12.703244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.714548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.767673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6a68a-d12d-4087-8499-854e0a77bca4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.801215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.801763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.802054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.802335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.803043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:12.803355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.804764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.805524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4394867-5a64-4bbc-b168-ca672f45dc86 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.813477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 41/83] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.814367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:12.817819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.841049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.841656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.841939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.842205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.909948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.911438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.913692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b2ac-9046-438d-85f4-7dca8f052333 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:12.919522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:12.947365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:12.948034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:12.948321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:12.948602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:12.950244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:12.951756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:12.953322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 42/84] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:12.954086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.019570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 43/85] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:13.019570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.028783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.060055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0ce5-51ac-41d1-bab8-c1b6210bf2bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.093886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.094499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.094783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.095067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.095760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.097219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.115817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e94581-a0e4-4b9a-b771-bb28ef797816 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.131819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 43/86] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:12 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:13.131819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.134212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.158808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.159471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.159790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.160069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.161272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.164650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 44/87] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:13.171136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.249527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2ac2b-87d1-4448-8a95-3f08999e9f31 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.279414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.285811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521d731-40ae-419b-bced-eb6cfe1713e8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.289984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.290662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.290937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.291212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.292398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.294064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.296159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 44/88] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:13.296868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.329542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.330460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.331150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.331589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.383022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.413322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.414969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.427132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0b23b-686b-43ec-b0ad-83ea62bcb564 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.466447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.467099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.467393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.467785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.468968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.470496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.472199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 45/89] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:13.474915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.540074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 45/90] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:13.541391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.554600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.616026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1147cc-1eed-45b5-93bd-715dbfa77df0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.634795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.659623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.659623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.659623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.659623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.659984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=af83574daa4c4563a4f76e72696e04f5) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:13.659984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.661254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.667875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de1cc-4a9b-4470-9fbd-e3fa5781ac38 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.671689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 46/91] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/role_assignments?user.id=af83574daa4c4563a4f76e72696e04f5&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=660e237685d342e8bb6bef00916bca0b => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:13.672673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:13.710440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.711821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.712254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.712672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.836809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.838321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.880762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662d97e-6c7e-4b7a-a403-36977eb757bc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:13.915913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:13.930152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:13.930767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:13.931049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:13.931323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:13.932129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] RBAC: Adding query filter params (role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=af83574daa4c4563a4f76e72696e04f5) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:13.932838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:13.934956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:13.940935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 47/92] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/role_assignments?user.id=af83574daa4c4563a4f76e72696e04f5&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:13.941901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:14.012388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:14.087152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 46/93] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:14.093827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:14.127347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ca89-e200-40d0-899c-f5e646c0b8af None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:14.165704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:14.166711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:14.167310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:14.167812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:14.169894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:14.171593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:14.173470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 48/94] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:13 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:14.180487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:14.272153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:14.326988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd54502-85e4-4d89-aef0-8b339e2b5879 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:14.358337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca881c40-4c22-48c6-9fff-8178f6a4ca64 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:14.390259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:14.390916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:14.391200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:14.391479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] PATH_INFO: `/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:14.392696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] RBAC: Authorizing `identity:get_project(project_id=ad9d95abbc8a4588ad09f6f4421e97bf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:14.394160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:14.399584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 47/95] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:14 2022] GET /identity/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf => generated 312 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:14.402122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:14.423957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:14.424499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:14.424778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:14.425049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:14.425905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=285daab328104f75886dde4ae3928fec) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:14.426215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:14.427757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0330027-fc8b-44b1-814a-cff05916ad90 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:14.440245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 49/96] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:14 2022] GET /identity/v3/role_assignments?user.id=285daab328104f75886dde4ae3928fec&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:14.455588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:14.547201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c75237-35a6-4e8d-a475-5e68d7b17405 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:14.604180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:14.608308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:14.608601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:14.608888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:14.609656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=ad9d95abbc8a4588ad09f6f4421e97bf, user.id=af83574daa4c4563a4f76e72696e04f5) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:14.609974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:14.611476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de38d88-61c7-406c-a45f-2bf0d63831a3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:14.621129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 48/97] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:14 2022] GET /identity/v3/role_assignments?user.id=af83574daa4c4563a4f76e72696e04f5&scope.project.id=ad9d95abbc8a4588ad09f6f4421e97bf&role.id=660e237685d342e8bb6bef00916bca0b => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:16.004166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edd4bf-8298-4de9-a6df-05ba54bbfd43 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:16.004719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edd4bf-8298-4de9-a6df-05ba54bbfd43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:16.004719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edd4bf-8298-4de9-a6df-05ba54bbfd43 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:16.006643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 50/98] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:16.013002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a45ae8-9c3b-4746-9adf-c9e670b01aed None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:16.013502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a45ae8-9c3b-4746-9adf-c9e670b01aed None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:16.013604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a45ae8-9c3b-4746-9adf-c9e670b01aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:16.034294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-81a45ae8-9c3b-4746-9adf-c9e670b01aed None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:16.035506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-81a45ae8-9c3b-4746-9adf-c9e670b01aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:16.120214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 49/99] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:16.218255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400127-4af3-4699-98e5-47a85d370900 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:16.218596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400127-4af3-4699-98e5-47a85d370900 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:16.218818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400127-4af3-4699-98e5-47a85d370900 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:16.220443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 51/100] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:16.234180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:16.234471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:16.234752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:16.259467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:16.260608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:16.300448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:16.353852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 50/101] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:16.681941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400127-4af3-4699-98e5-47a85d370900 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:16.810709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400127-4af3-4699-98e5-47a85d370900 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:16.811383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:16.846362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:16.865700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e5b1-b602-4633-bdba-40c96b612fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:16.867383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e5b1-b602-4633-bdba-40c96b612fce None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:16.867704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e5b1-b602-4633-bdba-40c96b612fce None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:16.867981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e5b1-b602-4633-bdba-40c96b612fce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:16.893378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68e5b1-b602-4633-bdba-40c96b612fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-be68e5b1-b602-4633-bdba-40c96b612fce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:16.901216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:16.902193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:16.904240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:16.905227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 52/102] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:16.906076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56916d99-3573-405b-96ed-e021958fca1c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:16.906362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56916d99-3573-405b-96ed-e021958fca1c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:16.906760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56916d99-3573-405b-96ed-e021958fca1c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:16.908299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 53/103] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:16.912637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56916d99-3573-405b-96ed-e021958fca1c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:17.005782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a6903-0627-4f00-9a84-429325062509 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:17.048180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0325-c2c1-4ce1-8f6b-8ce5f35ccf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:17.048951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0325-c2c1-4ce1-8f6b-8ce5f35ccf14 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.049387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0325-c2c1-4ce1-8f6b-8ce5f35ccf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.049666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0325-c2c1-4ce1-8f6b-8ce5f35ccf14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.051861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c0325-c2c1-4ce1-8f6b-8ce5f35ccf14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:17.053395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c0325-c2c1-4ce1-8f6b-8ce5f35ccf14 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:17.055239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 51/104] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:17.056246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.056530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.056851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.078388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:17.079522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:17.111934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:17.124926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56916d99-3573-405b-96ed-e021958fca1c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:17.164204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 52/105] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:17.165555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:17.214853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:17.215534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.215841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.216112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.216900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:17.217211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:17.218729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:17.237559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 54/106] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:17.247569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:17.335584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a87431-e448-4780-b052-b5b6236d5521 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:17.347997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:17.397693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:17.398328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.398612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.398938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.399619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:17.401677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:17.448244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 53/107] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:15:17 2022] POST /identity/v3/projects => generated 302 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:17.488085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d3910-a093-4776-8641-c7b80e223525 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:17.558597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaee3a-7953-4615-aed4-68aafff682be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:17.559876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaee3a-7953-4615-aed4-68aafff682be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.560218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaee3a-7953-4615-aed4-68aafff682be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.560494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaee3a-7953-4615-aed4-68aafff682be None admin] PATH_INFO: `/v3/users/62369218e24a4f00a6e92ec867a0a8fb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.582275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaee3a-7953-4615-aed4-68aafff682be None admin] RBAC: Authorizing `identity:get_user(user_id=62369218e24a4f00a6e92ec867a0a8fb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:17.583749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaee3a-7953-4615-aed4-68aafff682be None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:17.585721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 55/108] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:17 2022] GET /identity/v3/users/62369218e24a4f00a6e92ec867a0a8fb => generated 289 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:17.592196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:17.617138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:17.642863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14076a-1950-461e-bdbd-fdc781edf987 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.643166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14076a-1950-461e-bdbd-fdc781edf987 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.643420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14076a-1950-461e-bdbd-fdc781edf987 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.645122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 56/109] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:17.647588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14076a-1950-461e-bdbd-fdc781edf987 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:17.732780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966a873-d262-49ef-bbc3-115330ad58b7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:17.770746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe124202-9884-47a0-a2f4-55942e6cdd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:17.771634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124202-9884-47a0-a2f4-55942e6cdd67 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.772084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124202-9884-47a0-a2f4-55942e6cdd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.772503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124202-9884-47a0-a2f4-55942e6cdd67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.786364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14076a-1950-461e-bdbd-fdc781edf987 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:17.790290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:17.831971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356681-9d64-4fb8-a0af-ee22eb000e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:17.832700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356681-9d64-4fb8-a0af-ee22eb000e01 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.832987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356681-9d64-4fb8-a0af-ee22eb000e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.833262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356681-9d64-4fb8-a0af-ee22eb000e01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.848749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec356681-9d64-4fb8-a0af-ee22eb000e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:17.853317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe124202-9884-47a0-a2f4-55942e6cdd67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ec356681-9d64-4fb8-a0af-ee22eb000e01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:17.854572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:17.855424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:17.857018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:17.858230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe124202-9884-47a0-a2f4-55942e6cdd67 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:17.861540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 57/110] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:17.864285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc99fcc-5d4f-4828-92aa-05e844f4a853 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.864577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc99fcc-5d4f-4828-92aa-05e844f4a853 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.864857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc99fcc-5d4f-4828-92aa-05e844f4a853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.883911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:17.885384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dcc99fcc-5d4f-4828-92aa-05e844f4a853 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:17.887784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dcc99fcc-5d4f-4828-92aa-05e844f4a853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:17.965974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 58/111] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:17.967047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c18794-b9f5-4a36-8ddf-cdb37ac6d8f7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:17.967335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c18794-b9f5-4a36-8ddf-cdb37ac6d8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:17.967655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c18794-b9f5-4a36-8ddf-cdb37ac6d8f7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:17.971650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 59/112] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:17.971650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c18794-b9f5-4a36-8ddf-cdb37ac6d8f7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:17.999087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 54/113] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 06:15:18.000405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243df304-b67c-4018-a6ea-02b99092c72f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.000693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243df304-b67c-4018-a6ea-02b99092c72f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.000966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243df304-b67c-4018-a6ea-02b99092c72f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.003666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 55/114] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:18.005015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.005475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.005972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.026013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:18.027458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:18.088307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.097389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c18794-b9f5-4a36-8ddf-cdb37ac6d8f7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.108835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 56/115] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:18.109503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.139133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.140634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.141520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.142226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.143089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:18.143592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:18.146344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e38b-b0a4-4e85-b680-a584d86a21e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:18.157724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 60/116] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:18.159086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.160464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.160976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.161459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.183467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:18.186191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:18.243571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f02e4c-c7c3-4917-901c-28645efb7298 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.280012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.280012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.280012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.280012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.280406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:18.280406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:18.280525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 57/117] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:18.281380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.337354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 61/118] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:18.339084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.388145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf1196-26c6-4d76-9f9c-eb3e7bc4ce47 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.412514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.424057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.424818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.425168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.425523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.432297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:18.434376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:18.436885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 58/119] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:18.437665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.507499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.524665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dff7ae-d466-4024-8b01-fe58e3742467 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.563848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.564937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.565396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.565843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.566765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] RBAC: Adding query filter params (role.id=73bcb9d47c704919b2f24af36135d40f, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=62369218e24a4f00a6e92ec867a0a8fb) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:18.567271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:18.568911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:18.576165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 62/120] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/role_assignments?user.id=62369218e24a4f00a6e92ec867a0a8fb&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=73bcb9d47c704919b2f24af36135d40f => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:18.578573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0d144-a412-4cf6-8e23-f9ca6e7dc971 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.584583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.626076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.627065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.627520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.628065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.642114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:18.648396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:18.649305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:18.650089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:18.652638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 59/121] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:18.654277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.719187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.768313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22382e01-3270-4c16-b217-6f54c76b6c06 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.795056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf16a13-38ca-4892-b094-c1d0c15ddb1b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.805114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.806013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.806432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.806867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.808171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:18.809919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:18.811914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 63/122] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:18.812811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.833816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.834396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.835089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.835377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.835682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.847957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:18.853271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:18.854298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 60/123] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:18.855137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:18.924456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239e92e-66fb-4758-816b-33df92f9f5c9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.961028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:18.961920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:18.962343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:18.962803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:18.963764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:18.964207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:18.966134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:18.966698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1635bef-b4df-4afd-bb71-83535a996728 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:18.968471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:18.979009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 64/124] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:18.980143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.005581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.006269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.006497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.006881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.018592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:19.023887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:19.024757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:19.025566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:19.030351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 61/125] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:19.031340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.073413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:19.096143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652c556-7c09-4675-aabe-5e1350321173 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.133788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.134866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.135459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.135973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc/users/af83574daa4c4563a4f76e72696e04f5/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.138735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] RBAC: Authorizing `identity:create_grant(project_id=127332367fa24e318828c23b1f6090cc, user_id=af83574daa4c4563a4f76e72696e04f5, role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.140534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.142240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abfda4-c31f-49c0-bc56-18449d4c9799 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.157846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 65/126] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:18 2022] PUT /identity/v3/projects/127332367fa24e318828c23b1f6090cc/users/af83574daa4c4563a4f76e72696e04f5/roles/8ace72a8232f495d8e03de411d3ec43b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:19.159155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.181805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.182309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.182595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.182871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.183566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:19.184533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.186353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.195659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 62/127] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:19.196538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.226718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:19.286193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711667f-b07d-4e2b-a911-c6d792643671 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.328158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.329343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.329873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.330340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.332066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.333825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.335786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 66/128] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:19.336980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.356947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:19.412894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420f0a1-bd00-4e66-92c3-65cea9392ffd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.462178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.463139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.463644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.464111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.465083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:19.465748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.467847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.474970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8884835-efc4-4410-9e97-aeb9e1bbec2f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.488245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 63/129] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:19.489313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.512629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.513527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.514038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.514425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.515726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.517444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.519318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 67/130] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:19.520258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.674014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:19.682279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:19.704646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ba97e-9256-4881-a8b5-92030c9c163c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.727959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dff58-601f-4449-8540-1efa87e64753 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.767959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.768615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.768898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.769174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] PATH_INFO: `/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.770473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] RBAC: Authorizing `identity:get_project(project_id=ad9d95abbc8a4588ad09f6f4421e97bf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.771941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.773564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 64/131] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:19.780865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.793358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:19.794069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:19.794396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:19.794678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:19.795862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:19.797302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:19.798877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 68/132] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:19.800711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:19.974103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e2f7b-a8c0-4a2d-8e80-4383ab0da85c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:19.991623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6758c02-a6ac-47a6-a369-065e0ce15588 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:20.015239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:20.016324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.017317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.017793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.019414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:20.021133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:20.022937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 65/133] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:19 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:20.023521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:20.038956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7e50b-a0ca-435e-92a8-9350ef769046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:20.039656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e50b-a0ca-435e-92a8-9350ef769046 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.039938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e50b-a0ca-435e-92a8-9350ef769046 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.040221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7e50b-a0ca-435e-92a8-9350ef769046 None admin] PATH_INFO: `/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf/users/af83574daa4c4563a4f76e72696e04f5/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.042357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7e50b-a0ca-435e-92a8-9350ef769046 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad9d95abbc8a4588ad09f6f4421e97bf, user_id=af83574daa4c4563a4f76e72696e04f5, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:20.043992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7e50b-a0ca-435e-92a8-9350ef769046 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:20.053509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:20.062809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 69/134] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:19 2022] PUT /identity/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf/users/af83574daa4c4563a4f76e72696e04f5/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:20.064825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3a87-93e5-45c0-8ecd-1a06574e94f5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.065198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3a87-93e5-45c0-8ecd-1a06574e94f5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.065522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad3a87-93e5-45c0-8ecd-1a06574e94f5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.067281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 70/135] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:20.068578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3a87-93e5-45c0-8ecd-1a06574e94f5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:20.072392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:20.151358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5c12b-1c62-409c-a365-3604cc352603 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:20.192311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc15ba-a737-4df1-af59-4a6acfae941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:20.193254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc15ba-a737-4df1-af59-4a6acfae941b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.193710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc15ba-a737-4df1-af59-4a6acfae941b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.194201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc15ba-a737-4df1-af59-4a6acfae941b None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.195747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc15ba-a737-4df1-af59-4a6acfae941b None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:20.197964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc15ba-a737-4df1-af59-4a6acfae941b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:20.201174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 66/136] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:20 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:20.202803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.203292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.203753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.232301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad3a87-93e5-45c0-8ecd-1a06574e94f5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:20.232961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:20.234337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:20.322259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 67/137] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:20.323709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:20.332284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:20.333061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.333409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.333752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc/users/af83574daa4c4563a4f76e72696e04f5/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.337170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] RBAC: Authorizing `identity:create_grant(project_id=127332367fa24e318828c23b1f6090cc, user_id=af83574daa4c4563a4f76e72696e04f5, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:20.338707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:20.351137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:20.374877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 71/138] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:20 2022] PUT /identity/v3/projects/127332367fa24e318828c23b1f6090cc/users/af83574daa4c4563a4f76e72696e04f5/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:20.447268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d811-9826-40da-81f1-3244b8d99a39 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:20.487002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:20.488013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.488475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.488925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc/users/285daab328104f75886dde4ae3928fec/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.491268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=127332367fa24e318828c23b1f6090cc, user_id=285daab328104f75886dde4ae3928fec, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:20.492979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:20.510604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 68/139] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:20 2022] PUT /identity/v3/projects/127332367fa24e318828c23b1f6090cc/users/285daab328104f75886dde4ae3928fec/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:20.648137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:20.735201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:20.789816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15693e52-1864-4f73-910f-04d4c2033739 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:20.839352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c144631-3f21-4ea3-a4dd-ec6b618d26f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:20.840143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c144631-3f21-4ea3-a4dd-ec6b618d26f2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:20.840547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c144631-3f21-4ea3-a4dd-ec6b618d26f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:20.840975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c144631-3f21-4ea3-a4dd-ec6b618d26f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:20.842260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c144631-3f21-4ea3-a4dd-ec6b618d26f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:20.843760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c144631-3f21-4ea3-a4dd-ec6b618d26f2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:20.845389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 72/140] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:20.850984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:20.949036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:20.970768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9f26-276c-44a0-b539-bce2702688e6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:21.019021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94562530-a6ce-4f09-aadf-2b0f1a9c8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:21.019679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562530-a6ce-4f09-aadf-2b0f1a9c8631 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:21.019969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562530-a6ce-4f09-aadf-2b0f1a9c8631 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:21.020244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562530-a6ce-4f09-aadf-2b0f1a9c8631 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:21.020910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94562530-a6ce-4f09-aadf-2b0f1a9c8631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:21.022365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94562530-a6ce-4f09-aadf-2b0f1a9c8631 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:21.053237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 69/141] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:20 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:21.428065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db338974-de5a-4798-b903-5e85812cc074 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:21.429076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db338974-de5a-4798-b903-5e85812cc074 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:21.429580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db338974-de5a-4798-b903-5e85812cc074 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:21.431129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 73/142] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:21.437266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0bb84-09c0-458d-9c3e-3951cca4a592 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:21.437711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0bb84-09c0-458d-9c3e-3951cca4a592 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:21.437892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0bb84-09c0-458d-9c3e-3951cca4a592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:21.472630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4eb0bb84-09c0-458d-9c3e-3951cca4a592 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:21.473952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4eb0bb84-09c0-458d-9c3e-3951cca4a592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:21.560627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 70/143] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:21.820952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee2556-7c30-4530-a3e3-7079c59cf136 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:21.821297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee2556-7c30-4530-a3e3-7079c59cf136 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:21.821476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee2556-7c30-4530-a3e3-7079c59cf136 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:21.822777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 74/144] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:21.828872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:21.829162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:21.829441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:21.858172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:21.859352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:21.930951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:21.939894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 71/145] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:22.008454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee2556-7c30-4530-a3e3-7079c59cf136 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.128958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee2556-7c30-4530-a3e3-7079c59cf136 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:22.160354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:22.165529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:22.166168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:22.166446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:22.166717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:22.178492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:22.183392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:22.184955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:22.185726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 75/146] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:22.192719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.363397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2c663-26ac-4787-93c1-c90d6d65701b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:22.407942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:22.408570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:22.408850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:22.409125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:22.409812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:22.410119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:22.411518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:22.422663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 72/147] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:22.428155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.531629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.567081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5ca1a-7b35-41cd-9a59-2af64ca7d3ad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:22.659492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:22.660102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:22.660381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:22.660653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] PATH_INFO: `/v3/users/62369218e24a4f00a6e92ec867a0a8fb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:22.661978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] RBAC: Authorizing `identity:get_user(user_id=62369218e24a4f00a6e92ec867a0a8fb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:22.663437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:22.665081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 76/148] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/users/62369218e24a4f00a6e92ec867a0a8fb => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:22.671276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:22.672294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.725827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b095f-94fc-4d6e-92dc-624fae89d695 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:22.764956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:22.765709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:22.765998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:22.766278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:22.778308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:22.786208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:22.787122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:22.788355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 73/149] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:22.792749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:22.794900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.898481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e0874-cc0c-4cd7-937f-8392215e55af None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:22.916182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166feb-a2d5-4894-9f2b-ac986d6c035a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:22.940016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:22.951506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:22.952190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:22.952481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:22.952759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:22.954291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:22.959578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:22.960220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:22.960521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:22.960798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:22.961482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:22.961791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:22.962742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:22.964317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:22.967357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 77/150] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:22.972688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:22.985617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 74/151] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:22.992596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:23.135337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98651ef9-ebee-4e82-9427-07e0ccd1a083 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:23.149535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa297613-55be-46db-8dc7-d661d9572b2b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:23.178996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:23.179579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:23.179864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:23.180137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:23.181270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:23.182720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:23.192549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 75/152] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:22 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:23.197222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:23.198930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:23.202707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:23.203395 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:23.203707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:23.203987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc/users/62369218e24a4f00a6e92ec867a0a8fb/roles/73bcb9d47c704919b2f24af36135d40f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:23.209075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=127332367fa24e318828c23b1f6090cc, user_id=62369218e24a4f00a6e92ec867a0a8fb, role_id=73bcb9d47c704919b2f24af36135d40f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:23.210550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:23.237692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 78/153] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:22 2022] PUT /identity/v3/projects/127332367fa24e318828c23b1f6090cc/users/62369218e24a4f00a6e92ec867a0a8fb/roles/73bcb9d47c704919b2f24af36135d40f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:15:23.314422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:23.389197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71393e-07b0-4891-a739-9a3b2ec4a4db None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:23.432929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:23.433519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:23.434407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:23.434892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:23.524768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:23.526280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:23.598561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:23.682758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 76/154] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:23.690306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:23.810289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b679f5-61f7-4374-8213-ba98877b43c9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:23.851169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5a09c-f1e4-44e8-bbf0-2d393099a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:23.851853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5a09c-f1e4-44e8-bbf0-2d393099a50d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:23.852134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5a09c-f1e4-44e8-bbf0-2d393099a50d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:23.852409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5a09c-f1e4-44e8-bbf0-2d393099a50d None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:23.853638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5a09c-f1e4-44e8-bbf0-2d393099a50d None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:23.855152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5a09c-f1e4-44e8-bbf0-2d393099a50d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:23.857391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 79/155] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:23 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:23.863214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:23.951501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d1d2-31f8-4352-a8bb-1524c58b52c9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:23.951828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d1d2-31f8-4352-a8bb-1524c58b52c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:23.952027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d1d2-31f8-4352-a8bb-1524c58b52c9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:23.953911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 80/156] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:23.971311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:23.972460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:23.972966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:24.007841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:24.009652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:24.019190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a14e5-ae1b-4e3e-9574-4e07d8e30634 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:24.066121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:24.067065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:24.068012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:24.068463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:24.069748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] RBAC: Adding query filter params (role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=af83574daa4c4563a4f76e72696e04f5) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:24.071993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:24.073470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:24.089778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 77/157] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:23 2022] GET /identity/v3/role_assignments?user.id=af83574daa4c4563a4f76e72696e04f5&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:24.097646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:24.122559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 81/158] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:24.536273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:24.552749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:24.683301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2eed-62fc-45fd-8a57-3b97860387ae None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:24.732129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4df-92bb-4e6b-9938-0b57659727ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:24.733152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4df-92bb-4e6b-9938-0b57659727ef None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:24.733638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4df-92bb-4e6b-9938-0b57659727ef None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:24.734115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4df-92bb-4e6b-9938-0b57659727ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:24.745518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2af4df-92bb-4e6b-9938-0b57659727ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0d2af4df-92bb-4e6b-9938-0b57659727ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:24.751085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:24.753762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:24.754615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:24.757192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 78/159] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:24.763728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:24.874908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07a6bd-43d3-438f-8f52-aa7cb1b38205 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:24.914419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:24.915399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:24.915913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:24.916393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:24.917280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:24.918306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:24.920053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:24.928518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fc76e-f047-4a92-8ee0-2eaecb504280 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:24.929152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fc76e-f047-4a92-8ee0-2eaecb504280 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:24.929458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 82/160] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:24.929869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fc76e-f047-4a92-8ee0-2eaecb504280 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:24.931456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 79/161] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:24.940841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:24.941684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ab887-6c73-4dc0-a5f5-6f6036845721 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:24.942192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ab887-6c73-4dc0-a5f5-6f6036845721 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:24.942722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ab887-6c73-4dc0-a5f5-6f6036845721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:24.970990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-164ab887-6c73-4dc0-a5f5-6f6036845721 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:24.972711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-164ab887-6c73-4dc0-a5f5-6f6036845721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:24.999582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:25.064376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 80/162] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:25.066080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4314ab0-a2b0-495f-941b-c5a807a8ba1a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:25.066648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4314ab0-a2b0-495f-941b-c5a807a8ba1a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:25.067186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4314ab0-a2b0-495f-941b-c5a807a8ba1a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:25.068679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 81/163] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:25.076548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:25.077101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:25.077578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:25.083834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecbaa6-6a8d-4ab2-aab8-dc03d45664e6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:25.085232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:25.103062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:25.104771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:25.152108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:25.152813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:25.153099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:25.153372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:25.154591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:25.156198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:25.157901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 83/164] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:24 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:25.163420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:25.182326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 82/165] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:25.316847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a84962-1d82-4682-928f-3a6e3d275563 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:25.343155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:25.363332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:25.363976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:25.364254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:25.364524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:25.459156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:25.460567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:25.598808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 84/166] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 06:15:25.605174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:25.628883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:25.651586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:25.763045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5974def-9146-4ff4-9627-de0e947d98b7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:25.811495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5f5ff-c455-4d13-835e-154d7fc1d146 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:25.824611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:25.833587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:25.834556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:25.835037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:25.835595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] PATH_INFO: `/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:25.837055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] RBAC: Authorizing `identity:get_project(project_id=ad9d95abbc8a4588ad09f6f4421e97bf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:25.838888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:25.844641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 83/167] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:25 2022] GET /identity/v3/projects/ad9d95abbc8a4588ad09f6f4421e97bf => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:25.845848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:25.865186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:25.865842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:25.866159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:25.866378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:25.892734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:25.897835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:25.900592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:25.902760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 85/168] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:25.904694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:25.961879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e9d43-b600-4ecd-b0dc-439f5f13e0cc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:25.999753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:26.000811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.001368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.001937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.023206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:26.030612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:26.031509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:26.032319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:26.035268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 84/169] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:26.044830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:26.103994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:26.143521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686d7f-d07d-4e4d-a324-3df4e41fdb6a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:26.168483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:26.231923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:26.232564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.232841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.233111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.233875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=ad9d95abbc8a4588ad09f6f4421e97bf, user.id=af83574daa4c4563a4f76e72696e04f5) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:26.234182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:26.235680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:26.257956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 86/170] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:25 2022] GET /identity/v3/role_assignments?user.id=af83574daa4c4563a4f76e72696e04f5&scope.project.id=ad9d95abbc8a4588ad09f6f4421e97bf&role.id=660e237685d342e8bb6bef00916bca0b => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:26.258660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:26.272489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243870cd-a757-4a42-b178-b5a2e4e80918 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:26.336282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:26.337313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.337785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.338255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.339129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:26.339856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:26.341484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efefe56-81b6-4faf-942f-108f70b4cc05 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:26.357095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 85/171] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:26.358781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2647ae-08fb-4dec-9a70-58dcc39f6181 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.359263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2647ae-08fb-4dec-9a70-58dcc39f6181 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.359766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2647ae-08fb-4dec-9a70-58dcc39f6181 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.361287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 86/172] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:26.368083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f400c-3931-4148-acae-547199e7b17a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.368675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f400c-3931-4148-acae-547199e7b17a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.369154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f400c-3931-4148-acae-547199e7b17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.408539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f86f400c-3931-4148-acae-547199e7b17a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:26.418868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f86f400c-3931-4148-acae-547199e7b17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:26.474743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3a14-d23d-44a3-be61-773a8661b3d2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:26.552034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:26.552669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.552945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.553212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.553890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:26.554188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:26.555578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:26.574797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 87/173] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:26.575476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:26.577068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 87/174] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:26.580920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:26.586833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f400c-3931-4148-acae-547199e7b17a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:26.661441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:26.715612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f400c-3931-4148-acae-547199e7b17a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:26.723906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6891a7-4e23-47cf-95c5-873d828c2ceb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:26.757922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a540a-ba5b-4420-9c1b-174b7c3be27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:26.758581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a540a-ba5b-4420-9c1b-174b7c3be27e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.758858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a540a-ba5b-4420-9c1b-174b7c3be27e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.759130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a540a-ba5b-4420-9c1b-174b7c3be27e None admin] PATH_INFO: `/v3/users/285daab328104f75886dde4ae3928fec` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.760304 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a540a-ba5b-4420-9c1b-174b7c3be27e None admin] RBAC: Authorizing `identity:get_user(user_id=285daab328104f75886dde4ae3928fec)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:26.761940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a540a-ba5b-4420-9c1b-174b7c3be27e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:26.763579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 88/175] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:26 2022] GET /identity/v3/users/285daab328104f75886dde4ae3928fec => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:26.764747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095a78e-f981-4b68-a5c0-9897785e38f9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.765026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095a78e-f981-4b68-a5c0-9897785e38f9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.765301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095a78e-f981-4b68-a5c0-9897785e38f9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.766530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 89/176] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:26.772114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.772364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.772642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.782998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:26.783998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:26.784499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:26.784970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] PATH_INFO: `/v3/users/af83574daa4c4563a4f76e72696e04f5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:26.786380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] RBAC: Authorizing `identity:get_user(user_id=af83574daa4c4563a4f76e72696e04f5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:26.788201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:26.790003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 88/177] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:26 2022] GET /identity/v3/users/af83574daa4c4563a4f76e72696e04f5 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:26.791001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:26.793869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:26.795099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:26.869057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 90/178] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:26.869874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:26.882914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:26.927111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:26.964093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c6909-f37f-457a-8dcd-f1cc199f6f69 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede03387-aa67-4756-8ee5-d39925f366a2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.015189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9705d-e3b5-4198-b9f5-63fba676127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.015871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9705d-e3b5-4198-b9f5-63fba676127b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.016147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9705d-e3b5-4198-b9f5-63fba676127b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.016413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9705d-e3b5-4198-b9f5-63fba676127b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.081350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.082245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.082930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.083207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.097690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9705d-e3b5-4198-b9f5-63fba676127b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.099397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9705d-e3b5-4198-b9f5-63fba676127b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.161540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.163386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.177897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.216154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 89/179] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:27.222038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf443bd-c709-4de2-b5d6-5ba327a6f2b0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.222333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf443bd-c709-4de2-b5d6-5ba327a6f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.222610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf443bd-c709-4de2-b5d6-5ba327a6f2b0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.222780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.224544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 90/180] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:27.225374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf443bd-c709-4de2-b5d6-5ba327a6f2b0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.285332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 91/181] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:27.287831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.332838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf443bd-c709-4de2-b5d6-5ba327a6f2b0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.366785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.367408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.367710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.367980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.378497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:27.383299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:27.384270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:27.385054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 91/182] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:27.385715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.389064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c32ff-772a-4dec-8b1b-f44028e1cc52 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.416507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.423821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47ce42-427a-49e0-8f08-f1a0f164048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.424444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ce42-427a-49e0-8f08-f1a0f164048e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.424718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ce42-427a-49e0-8f08-f1a0f164048e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.424984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ce42-427a-49e0-8f08-f1a0f164048e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.426066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47ce42-427a-49e0-8f08-f1a0f164048e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.427504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47ce42-427a-49e0-8f08-f1a0f164048e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.429078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 92/183] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:27.429078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cf9df-303f-4877-972b-42eae80e8b7d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.429078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cf9df-303f-4877-972b-42eae80e8b7d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.429078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cf9df-303f-4877-972b-42eae80e8b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.448285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-124cf9df-303f-4877-972b-42eae80e8b7d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:27.449508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-124cf9df-303f-4877-972b-42eae80e8b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:27.464128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.492234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bc235-2bf4-4a41-93b0-3764895a3f00 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.518814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 93/184] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:27.520173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a06dc-72a7-4a98-b996-7eb31133e7b8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.520464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a06dc-72a7-4a98-b996-7eb31133e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.520733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a06dc-72a7-4a98-b996-7eb31133e7b8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.522119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 94/185] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:27.522729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a06dc-72a7-4a98-b996-7eb31133e7b8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.532563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.533629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.533984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.535209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.536763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.538631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 92/186] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:27.539307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.643351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a06dc-72a7-4a98-b996-7eb31133e7b8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.646501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.649471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00700d75-b965-4415-ab30-40bb98ed3760 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.679505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.680252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.680541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.680817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.681936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.683506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.685427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 95/187] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:27.686118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.689565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.690287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.690569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.690859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.691525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:27.691855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.693205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd5d00-5f8d-49ff-88f4-a4e400c84108 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.702553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 93/188] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:27.705130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.705767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.706529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.709476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.728452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:27.729639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:27.795990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfba20-f7d3-4fb8-8de5-e6f14bd6ec84 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.800212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 94/189] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:27.801326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.832165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.832784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.833063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.833337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.834011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.835449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.857900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 96/190] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:27 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:27.858732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:27.914290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.931257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3b5a-798c-425f-8331-2f31da1adbd4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:27.962899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:27.973285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:27.974322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:27.974454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:27.974720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:27.975643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=285daab328104f75886dde4ae3928fec) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:27.976003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:27.977564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:27.985211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 95/191] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/role_assignments?user.id=285daab328104f75886dde4ae3928fec&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:27.986076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.011513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95551fb6-f4f8-4e8d-b202-8e4c8a873f70 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.064706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.065720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.066207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.066701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.067706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=af83574daa4c4563a4f76e72696e04f5) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:28.068390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:28.070120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:28.080020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 97/192] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/role_assignments?user.id=af83574daa4c4563a4f76e72696e04f5&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=660e237685d342e8bb6bef00916bca0b => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:28.081562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.157996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae55f5d-b8cc-4342-b8a7-7b402563e255 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.207423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.208104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.208389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.208669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] PATH_INFO: `/v3/users/bc3414f53fe14c48a6d2454471e74407` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.233186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] RBAC: Authorizing `identity:get_user(user_id=bc3414f53fe14c48a6d2454471e74407)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:28.234998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:28.238331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 96/193] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:27 2022] GET /identity/v3/users/bc3414f53fe14c48a6d2454471e74407 => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:28.239955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.251421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c2db-a804-4cd3-8f71-ed9df0b5a267 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.263284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:28.303507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.304432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.304862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.305308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.309512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:28.320692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:28.327048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:28.328022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:28.328847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:28.329478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 98/194] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:28.331360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.410088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e319ed-392b-4e94-a215-327066f5f8c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.466893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.467471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.467761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.468034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.471249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fb751-efed-4670-993b-09d6e7f855e0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.479922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:28.486062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:28.486977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 97/195] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:28.487847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.519017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.520107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.520530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.520941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.576140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:28.595964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:28.602990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:28.604995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:28.619738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f14ff-adf6-429f-a705-4fbb5a3f6930 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.658480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.659516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.659932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.660210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.660953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:28.661663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:28.663444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:28.675603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 98/196] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:28.676734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.739803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 99/197] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:28.741364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.812954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9bd99-cfd1-4820-8eaa-a81390bcdd85 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.856013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.856895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.857331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.857763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.858679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:28.859338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:28.859947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:28.862217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:28.865236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:28.865980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2406f-39f4-45a7-8307-2f6b61dc2bfc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:28.875781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 99/198] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:28.877004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.914529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:28.915175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:28.915458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:28.915760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] PATH_INFO: `/v3/users/bc3414f53fe14c48a6d2454471e74407` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:28.917066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] RBAC: Authorizing `identity:get_user(user_id=bc3414f53fe14c48a6d2454471e74407)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:28.919414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:28.921048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 100/199] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/users/bc3414f53fe14c48a6d2454471e74407 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:28.921909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:28.998820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b209-d311-4bcd-a644-773e7df00077 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.041957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.043029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.043521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.044047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.045981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.046572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a068ae-8017-4970-b1d2-6c6a7dba6370 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.047949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.050154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 100/200] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:29.051167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:29.095833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.096532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.096862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.097139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] PATH_INFO: `/v3/users/62369218e24a4f00a6e92ec867a0a8fb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.098897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] RBAC: Authorizing `identity:get_user(user_id=62369218e24a4f00a6e92ec867a0a8fb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.100396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.103043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 101/201] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:28 2022] GET /identity/v3/users/62369218e24a4f00a6e92ec867a0a8fb => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:29.107440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:29.128118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:29.157299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:29.183853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a39df-3162-4d17-8b64-7d4ced0ede8f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.222777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.223912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.224399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.224927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.253801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eda03-dd16-4351-a421-7ab980bc394f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.294172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.295111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.295634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.296130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.297048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=bc3414f53fe14c48a6d2454471e74407) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:29.297554 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.299302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.306189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 102/202] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:29 2022] GET /identity/v3/role_assignments?user.id=bc3414f53fe14c48a6d2454471e74407&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:29.306710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.307655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:29.308472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.450760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:29.473828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:29.482920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe026-66d1-4096-98d3-62768709df8a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.523272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.524607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.525108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.525582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.541941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 101/203] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:29.548896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:29.671173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e4187-0423-4cc0-906c-aa639134c69e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.697603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.699156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.713461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.714102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.714914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.715231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.717203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.719803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.721628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 102/204] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:29 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:29.727474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:29.779681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:29.885593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:29.885593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 103/205] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:29.887802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:29.903790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad43e09-e82b-4914-a677-77c2a749a340 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:29.961162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:29.961778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:29.962052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:29.962318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:29.963054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] RBAC: Adding query filter params (role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=bc3414f53fe14c48a6d2454471e74407) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:29.963355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:29.968364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:29.979697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 103/206] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:29 2022] GET /identity/v3/role_assignments?user.id=bc3414f53fe14c48a6d2454471e74407&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:30.041480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab035b-b06d-47c5-9dc9-d77f3ee3b762 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:30.081822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba1cfd6-f1c6-4fd6-a821-ef75734ff1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:30.082466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1cfd6-f1c6-4fd6-a821-ef75734ff1cd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:30.082740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1cfd6-f1c6-4fd6-a821-ef75734ff1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:30.083008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba1cfd6-f1c6-4fd6-a821-ef75734ff1cd None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:30.084216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1cfd6-f1c6-4fd6-a821-ef75734ff1cd None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:30.085770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba1cfd6-f1c6-4fd6-a821-ef75734ff1cd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:30.087640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 104/207] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:29 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:30.096723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:30.274440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:30.280686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1bdb5-1e3a-4af0-8fee-cddb2d880bdd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:30.336441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:30.336984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:30.337269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:30.337548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:30.338299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] RBAC: Adding query filter params (role.id=73bcb9d47c704919b2f24af36135d40f, scope.project.id=127332367fa24e318828c23b1f6090cc, user.id=62369218e24a4f00a6e92ec867a0a8fb) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:30.338612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:30.340666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15984791-982c-43b7-8728-ed5ce34b58e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:30.352433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 104/208] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:30 2022] GET /identity/v3/role_assignments?user.id=62369218e24a4f00a6e92ec867a0a8fb&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=73bcb9d47c704919b2f24af36135d40f => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:30.620209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed55b3-f70c-4cda-ba2e-0bfbf45ce5be None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:30.620509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed55b3-f70c-4cda-ba2e-0bfbf45ce5be None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:30.621213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed55b3-f70c-4cda-ba2e-0bfbf45ce5be None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:30.622733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 105/209] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:30.628598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:30.628926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:30.629168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:30.652833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:30.654049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:30.670420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8668-d4c8-4395-a3a6-a9e097c54912 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:30.671049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8668-d4c8-4395-a3a6-a9e097c54912 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:30.671527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8668-d4c8-4395-a3a6-a9e097c54912 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:30.673094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 106/210] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:30.678999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:30.679519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:30.680032 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:30.701617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:30.702804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:30.740726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 105/211] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:30.782428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:30.793632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 107/212] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:31.046392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.085242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.154644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:31.166556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae277d72-226a-467d-b38b-d759634869c5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:31.201692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:31.202336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.202619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.202891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.213638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:31.219165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:31.220160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:31.221149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 106/213] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:31.227625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.233797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3064f1-391f-4680-b5af-6e183ee87448 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:31.280278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:31.281577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.281998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.282405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.283794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:31.286038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:31.288126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 108/214] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:31.293781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.355897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78610e2-0e39-4e11-b38d-ffb991d6cbd5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:31.372562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:31.394836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:31.395825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.396340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.396812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.397673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:31.398164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:31.399765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:31.408307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cd23f-9ed4-47b1-ad1d-bea363c292a3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:31.413789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 107/215] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:31.414922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.431344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:31.446034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:31.446615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.446901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.447178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.448021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:31.449707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:31.475576 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 109/216] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:31 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:31.540401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06e615-f4cb-477c-ba3f-127a1fcc2a04 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:31.587979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde6e2-a499-41f0-b0d4-1ce7eb032b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:31.588936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde6e2-a499-41f0-b0d4-1ce7eb032b70 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.589552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde6e2-a499-41f0-b0d4-1ce7eb032b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.590016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde6e2-a499-41f0-b0d4-1ce7eb032b70 None admin] PATH_INFO: `/v3/users/119aab44bbdc4f93aa8a4a928122b980` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.613579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde6e2-a499-41f0-b0d4-1ce7eb032b70 None admin] RBAC: Authorizing `identity:get_user(user_id=119aab44bbdc4f93aa8a4a928122b980)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:31.615212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde6e2-a499-41f0-b0d4-1ce7eb032b70 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:31.617030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 108/217] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:31 2022] GET /identity/v3/users/119aab44bbdc4f93aa8a4a928122b980 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:31.623202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.731495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ab79c-5f73-454f-8c2f-66bd01e9c1a8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:31.749707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:31.769063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:31.769885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.770332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.770747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.855631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:31.857124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:31.926220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f85429-5bf8-4549-9163-bca5dcff1d6f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.926556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f85429-5bf8-4549-9163-bca5dcff1d6f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.926777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f85429-5bf8-4549-9163-bca5dcff1d6f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.928426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 109/218] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:31.934952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:31.935230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:31.935503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:31.961725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:31.962933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:31.971729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 110/219] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:31.978124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:31.998402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:32.051002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:32.056370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 110/220] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:32.099171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b0299-209a-4b47-a791-ca6684b4d2b9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:32.137241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:32.137884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.138167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.138437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.139603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:32.141057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:32.143042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 111/221] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:31 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:32.148613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:32.287821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6717c9-4d03-47be-b8dd-6f9f45398ad9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:32.333319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:32.333938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.334216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.334485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.335223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=119aab44bbdc4f93aa8a4a928122b980) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:32.335529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:32.336976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753862f6-ffd4-4570-b8c3-e515d7766a9e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:32.348014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 111/222] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:32 2022] GET /identity/v3/role_assignments?user.id=119aab44bbdc4f93aa8a4a928122b980&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=660e237685d342e8bb6bef00916bca0b => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:32.370268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:32.516472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:32.535911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dad57-9749-497f-9ecd-10c42640fc84 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:32.559850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78143f1-0fe9-48d7-90d7-d5c4d6b95b15 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.560521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78143f1-0fe9-48d7-90d7-d5c4d6b95b15 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.561016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78143f1-0fe9-48d7-90d7-d5c4d6b95b15 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.562542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 112/223] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:32.573639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.574346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.574816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.575003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:32.575641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.575930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.576207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.590038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:32.595636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:32.595841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:32.596686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:32.597484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 112/224] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:32.599856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:32.603836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:32.672608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:32.686279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 113/225] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:32.717655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c0fa-7993-41eb-bddb-5ba585b37490 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:32.754321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:32.754949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.755232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.755506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.756348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:32.756652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:32.758038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:32.770899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 113/226] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:32.776286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:32.914108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e428cd5-2561-48c4-81e8-48c21cfb6c8e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:32.960768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:32.961756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:32.962271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:32.962744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] PATH_INFO: `/v3/users/bc3414f53fe14c48a6d2454471e74407` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:32.964209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] RBAC: Authorizing `identity:get_user(user_id=bc3414f53fe14c48a6d2454471e74407)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:32.965975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:32.968153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 114/227] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:32 2022] GET /identity/v3/users/bc3414f53fe14c48a6d2454471e74407 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:32.973738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:32.994802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:33.125265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfeaa-79f1-4e29-aafa-77eefe4f758e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:33.174180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:33.174753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:33.175036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:33.175316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:33.176505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:33.178033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:33.180021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 114/228] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:32 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:33.188863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:33.209294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:33.314981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:33.334594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:33.360190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06aa4b-503d-4a22-96ce-294a04371ceb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:33.396644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:33.397763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:33.398237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:33.398699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:33.410784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:33.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:33.417694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:33.418517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:33.421172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 115/229] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:33.421855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a753b-88a6-4b79-a3d5-519d6b276181 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:33.426829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:33.535240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937ba09-6acb-4954-85d7-d3a2564afd93 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:33.545405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:33.545405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:33.545405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:33.545669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649/users/bc3414f53fe14c48a6d2454471e74407/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:33.552373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] RBAC: Authorizing `identity:create_grant(project_id=13c11b499ec641ce8a2a829378a48649, user_id=bc3414f53fe14c48a6d2454471e74407, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:33.561122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:33.586155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 115/230] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:33 2022] PUT /identity/v3/projects/13c11b499ec641ce8a2a829378a48649/users/bc3414f53fe14c48a6d2454471e74407/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:33.602476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:33.603567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:33.604082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:33.604551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:33.606390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:33.606924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:33.608585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:33.611744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:33.626983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 116/231] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:33.632407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:33.771983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81edde-7893-403f-b66b-96c6d06a853d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:33.805288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:33.818919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:33.819970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:33.820249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:33.820523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] PATH_INFO: `/v3/users/bc3414f53fe14c48a6d2454471e74407` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:33.821741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] RBAC: Authorizing `identity:get_user(user_id=bc3414f53fe14c48a6d2454471e74407)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:33.823988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:33.830592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 116/232] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:33 2022] GET /identity/v3/users/bc3414f53fe14c48a6d2454471e74407 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:33.832046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:33.941562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09380eea-ec45-4664-a70a-a68880a89f90 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:33.983764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fdf94-2bf0-41e6-9932-de901d7b72c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:33.984780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fdf94-2bf0-41e6-9932-de901d7b72c8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:33.985262 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fdf94-2bf0-41e6-9932-de901d7b72c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:33.985723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fdf94-2bf0-41e6-9932-de901d7b72c8 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:33.987069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fdf94-2bf0-41e6-9932-de901d7b72c8 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:33.988737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fdf94-2bf0-41e6-9932-de901d7b72c8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:33.990575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 117/233] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:33 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:33.996597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:34.125061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26f0b7-9702-4d43-bffc-3d388ddd9046 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:34.170320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b2917-8daa-4243-a2d9-75b289234c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:34.170978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b2917-8daa-4243-a2d9-75b289234c87 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:34.171247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b2917-8daa-4243-a2d9-75b289234c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:34.171524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b2917-8daa-4243-a2d9-75b289234c87 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649/users/bc3414f53fe14c48a6d2454471e74407/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:34.174816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b2917-8daa-4243-a2d9-75b289234c87 None admin] RBAC: Authorizing `identity:create_grant(project_id=13c11b499ec641ce8a2a829378a48649, user_id=bc3414f53fe14c48a6d2454471e74407, role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:34.176306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b2917-8daa-4243-a2d9-75b289234c87 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:34.206902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 117/234] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:33 2022] PUT /identity/v3/projects/13c11b499ec641ce8a2a829378a48649/users/bc3414f53fe14c48a6d2454471e74407/roles/8ace72a8232f495d8e03de411d3ec43b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:34.340882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3474c53-3c81-4702-b670-c72fef696f20 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:34.341900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3474c53-3c81-4702-b670-c72fef696f20 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:34.342373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3474c53-3c81-4702-b670-c72fef696f20 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:34.344312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 118/235] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:34.353853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feee505-bed9-41e9-bffd-2e357eb3a1b6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:34.354373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feee505-bed9-41e9-bffd-2e357eb3a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:34.354373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feee505-bed9-41e9-bffd-2e357eb3a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:34.375400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5feee505-bed9-41e9-bffd-2e357eb3a1b6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:34.376610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5feee505-bed9-41e9-bffd-2e357eb3a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:34.436229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:34.460805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 118/236] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:34.786456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3474c53-3c81-4702-b670-c72fef696f20 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:34.836861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3bad0-ae2e-4cf2-bb46-c9850585d847 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:34.837490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3bad0-ae2e-4cf2-bb46-c9850585d847 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:34.837960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3bad0-ae2e-4cf2-bb46-c9850585d847 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:34.841542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 119/237] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:34.845835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1e09-8dd2-461a-8ed7-1b0de70f8cf6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:34.846340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1e09-8dd2-461a-8ed7-1b0de70f8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:34.846811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1e09-8dd2-461a-8ed7-1b0de70f8cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:34.870824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:34.885754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b8ec1e09-8dd2-461a-8ed7-1b0de70f8cf6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:34.887791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b8ec1e09-8dd2-461a-8ed7-1b0de70f8cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:34.958697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3474c53-3c81-4702-b670-c72fef696f20 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:34.991885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 120/238] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:35.015275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264437-202a-4678-b471-db9beecb49b0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.015596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264437-202a-4678-b471-db9beecb49b0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.015852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264437-202a-4678-b471-db9beecb49b0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.017205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 121/239] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:35.023779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.024061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.024334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.042818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:35.043375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.043679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.043953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.045102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:35.046553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:35.048157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 119/240] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:35.053728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.063281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:35.064325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:35.191467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 122/241] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:35.225150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d937e-c7fa-455d-92a3-cf0820894a8d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:35.265145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:35.266031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.266493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.266941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.267766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:35.269342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:35.277876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:35.279696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.306135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 120/242] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:35 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:35.357597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:35.436819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a6ca8-c44e-45f8-92cc-60238347f672 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:35.480790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:35.481781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.482212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.482650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.501936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:35.507301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:35.508290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:35.509114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:35.511736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 123/243] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:35.521677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.635322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.650931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8be96-4f7d-46b6-90dc-168505440c40 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:35.689071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:35.689797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.690090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.690466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.691160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:35.691472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:35.692958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:35.702826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 121/244] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:35.709550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.764041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:35.767989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c43b1-f970-448f-a874-9ce99ded2295 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:35.793237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:35.820345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:35.821640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.822201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.822797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.836824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:35.843190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:35.844096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:35.847352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 124/245] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:35.857537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.868778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725e8d6-819b-4ea9-a7f4-a0b92dbba12f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:35.905720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:35.906369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:35.906653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:35.906928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] PATH_INFO: `/v3/users/46085f4526b241519ce46c9733f2755c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:35.922710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] RBAC: Authorizing `identity:get_user(user_id=46085f4526b241519ce46c9733f2755c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:35.924537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:35.928161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 122/246] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:35 2022] GET /identity/v3/users/46085f4526b241519ce46c9733f2755c => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:35.932395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:35.986952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8b07f-681e-49be-9b3f-52d8663824a5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.030351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.030971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.031357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.031665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.032352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:36.032739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.034197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.044487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 125/247] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:36.051059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:36.057550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9224b0-d22c-496a-b093-15a6bc5893e1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.070435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:36.084034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:36.101362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776b298-5a6e-4607-b3d4-e5bb5ee7a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.101982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776b298-5a6e-4607-b3d4-e5bb5ee7a77b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.102259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776b298-5a6e-4607-b3d4-e5bb5ee7a77b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.102975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776b298-5a6e-4607-b3d4-e5bb5ee7a77b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.189845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8449838-312d-46fc-ab41-060f8da41feb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.211592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776b298-5a6e-4607-b3d4-e5bb5ee7a77b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.213207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776b298-5a6e-4607-b3d4-e5bb5ee7a77b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.228372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.229020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.229303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.230089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] PATH_INFO: `/v3/users/119aab44bbdc4f93aa8a4a928122b980` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.231353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] RBAC: Authorizing `identity:get_user(user_id=119aab44bbdc4f93aa8a4a928122b980)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.233259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.235496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 126/248] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:36 2022] GET /identity/v3/users/119aab44bbdc4f93aa8a4a928122b980 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:36.248548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:36.435109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:36.441414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 123/249] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:36.442773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bc1b2-e908-42e9-8b06-bf8f80778672 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.443054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bc1b2-e908-42e9-8b06-bf8f80778672 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.443328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bc1b2-e908-42e9-8b06-bf8f80778672 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.444919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 124/250] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:36.446383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bc1b2-e908-42e9-8b06-bf8f80778672 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:36.458179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651490-d0a8-4b4c-85f3-5592619be254 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.475058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:36.529831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77924bd4-a647-4129-80cd-64b0c58368a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.530484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77924bd4-a647-4129-80cd-64b0c58368a1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.530761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77924bd4-a647-4129-80cd-64b0c58368a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.531032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77924bd4-a647-4129-80cd-64b0c58368a1 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.532632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77924bd4-a647-4129-80cd-64b0c58368a1 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.534154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77924bd4-a647-4129-80cd-64b0c58368a1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.548678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 127/251] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:36 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:36.550250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.550682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.551099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.576905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:36.578850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:36.663269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bc1b2-e908-42e9-8b06-bf8f80778672 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.684057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 128/252] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:36.684909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:36.727123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.727815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.728095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.728363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.730204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.732211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.733973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 125/253] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:36 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:36.740533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:36.808015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:36.827653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ab50f-5e4b-4707-96de-0c04e660da5b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.895926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:36.898727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.899418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.899846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.900125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649/users/119aab44bbdc4f93aa8a4a928122b980/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.905030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] RBAC: Authorizing `identity:create_grant(project_id=13c11b499ec641ce8a2a829378a48649, user_id=119aab44bbdc4f93aa8a4a928122b980, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.906549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.914862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fca784-996b-4f10-aa1a-bc7283202286 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:36.929780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 129/254] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:36 2022] PUT /identity/v3/projects/13c11b499ec641ce8a2a829378a48649/users/119aab44bbdc4f93aa8a4a928122b980/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:36.960605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:36.961280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:36.961566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:36.961839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:36.962639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] RBAC: Adding query filter params (role.id=73bcb9d47c704919b2f24af36135d40f, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=46085f4526b241519ce46c9733f2755c) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:36.962951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:36.964737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b10ca-baca-45d6-9d94-0a1343aff191 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:36.971274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 126/255] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:36 2022] GET /identity/v3/role_assignments?user.id=46085f4526b241519ce46c9733f2755c&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=73bcb9d47c704919b2f24af36135d40f => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:37.156216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:37.287746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2afde-8655-4c90-a4c3-aabcf2ba498a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:37.289188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce2b6-7f7d-4bf3-8a32-8e439653b6a5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:37.289474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce2b6-7f7d-4bf3-8a32-8e439653b6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:37.289749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce2b6-7f7d-4bf3-8a32-8e439653b6a5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:37.292724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 127/256] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:37.301864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:37.302148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:37.302426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:37.332289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:37.333283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:37.333762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:37.334298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:37.337838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:37.339681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:37.348357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:37.353605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:37.354608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:37.356186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:37.358324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 130/257] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:37.364065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:37.377014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:37.470280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 128/258] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:37.511665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2853a-bf44-4f93-83b8-97f811782e64 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:37.550275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:37.550874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:37.551149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:37.551429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:37.552111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:37.552411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:37.553782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:37.563454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 131/259] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:37.570616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:37.646798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:37.679909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3ebbb-7526-4606-8fe1-12a039b853f3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:37.717023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:37.717668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:37.717943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:37.718215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] PATH_INFO: `/v3/users/bc3414f53fe14c48a6d2454471e74407` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:37.719335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] RBAC: Authorizing `identity:get_user(user_id=bc3414f53fe14c48a6d2454471e74407)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:37.720869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:37.722381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 129/260] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:37 2022] GET /identity/v3/users/bc3414f53fe14c48a6d2454471e74407 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:37.727867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:37.801745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:37.835593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743f670-1113-42f5-8f87-f6bd22f0c220 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:37.872110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:37.872727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:37.873007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:37.873275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:37.947811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:37.949718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:38.038681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:38.079616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 132/261] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:38.085333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:38.125463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:38.251418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631a020-bb58-4ed0-aa6a-4dd223a912da None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:38.287884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:38.288525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.288856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.289070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.290189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:38.291667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:38.294508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 130/262] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:38 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:38.300152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:38.311677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:38.317112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680ec64-ca76-421f-b634-a47e9e46bbab None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:38.384826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:38.385480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.386296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.386571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.408835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:38.425940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:38.428116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:38.430260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 133/263] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:38.436634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:38.470851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad7f2a-ad31-41f4-8d87-145010fb7142 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:38.498183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:38.552905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:38.553827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.554302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.554789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.555750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=bc3414f53fe14c48a6d2454471e74407) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:38.556274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:38.558495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:38.576979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 131/264] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:38 2022] GET /identity/v3/role_assignments?user.id=bc3414f53fe14c48a6d2454471e74407&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:38.660208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a991f-cbc8-4614-8f01-d5c99f785a76 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:38.713439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:38.714631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.714913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.715216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.715908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:38.716216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:38.717603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f564bc5-f79f-4f63-9c97-8dcf7ed24d75 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:38.731923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 134/265] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:38.737360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:38.817664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad7e5-7c69-411e-9bef-58c3255b99ff None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.817959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad7e5-7c69-411e-9bef-58c3255b99ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.818147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad7e5-7c69-411e-9bef-58c3255b99ff None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.819702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 135/266] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:38.825803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.826093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.826475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.850071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:38.860299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:38.861557 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:38.903939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03997e-787a-42e6-b601-c98235ed2faf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:38.968042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:38.969035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:38.969533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:38.971081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] PATH_INFO: `/v3/users/bc3414f53fe14c48a6d2454471e74407` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:38.972546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] RBAC: Authorizing `identity:get_user(user_id=bc3414f53fe14c48a6d2454471e74407)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:38.975199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:38.977954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 132/267] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:38 2022] GET /identity/v3/users/bc3414f53fe14c48a6d2454471e74407 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:38.984393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:39.008161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 136/268] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:39.120095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b827db-faaf-4442-a913-d2dfcf39d910 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:39.164426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:39.165426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.165934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.166419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.180060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:39.252311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:39.254117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:39.378754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 133/269] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 06:15:39.384274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:39.421469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:39.516760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1c5b6-783e-4fda-bac8-afe68526f924 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:39.576524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428470-5353-4342-8dab-817c2b12c1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:39.577236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428470-5353-4342-8dab-817c2b12c1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.577524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428470-5353-4342-8dab-817c2b12c1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.577801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428470-5353-4342-8dab-817c2b12c1e9 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.579704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428470-5353-4342-8dab-817c2b12c1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:39.581417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428470-5353-4342-8dab-817c2b12c1e9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:39.583361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 137/270] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:39 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:39.589718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:39.602952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13116fbd-5c50-47fc-811b-6582a726f925 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.603263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13116fbd-5c50-47fc-811b-6582a726f925 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.603499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13116fbd-5c50-47fc-811b-6582a726f925 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.606536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 138/271] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:39.615071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.615349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.615657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.645110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:39.646589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:39.664312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:39.736776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791edff4-7fa1-4d6e-9f7a-f4166737cff9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:39.743348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 139/272] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:39.744895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:39.780883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:39.791694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:39.793794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.795717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.796188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.797104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] RBAC: Adding query filter params (role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=bc3414f53fe14c48a6d2454471e74407) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:39.798389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:39.800775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10713c0d-077f-4a5c-8ba0-004d580b5d94 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:39.815931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 134/273] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:39 2022] GET /identity/v3/role_assignments?user.id=bc3414f53fe14c48a6d2454471e74407&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:39.931738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a4a5-1b36-497f-bdff-431aa851d258 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:39.960859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66cdb-f10c-49ed-8c05-1e1f8e5596c2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.961140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66cdb-f10c-49ed-8c05-1e1f8e5596c2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.961419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66cdb-f10c-49ed-8c05-1e1f8e5596c2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.962802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 135/274] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:39.969148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b40e6d-019b-48bf-9455-a0924186150c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:39.969433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b40e6d-019b-48bf-9455-a0924186150c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:39.969706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b40e6d-019b-48bf-9455-a0924186150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:39.997377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e0b40e6d-019b-48bf-9455-a0924186150c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:39.998990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e0b40e6d-019b-48bf-9455-a0924186150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:39.999611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171cd00-f5bb-434c-8956-2f499320685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.000278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171cd00-f5bb-434c-8956-2f499320685a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.000567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171cd00-f5bb-434c-8956-2f499320685a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.000844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171cd00-f5bb-434c-8956-2f499320685a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.002061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171cd00-f5bb-434c-8956-2f499320685a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:40.004786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171cd00-f5bb-434c-8956-2f499320685a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:40.006766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 140/275] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:40.007972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1009a-310b-4ed4-829b-87069c25a6b5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.008318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1009a-310b-4ed4-829b-87069c25a6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.008601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1009a-310b-4ed4-829b-87069c25a6b5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.010303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 141/276] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:40.013917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1009a-310b-4ed4-829b-87069c25a6b5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.092526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 136/277] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:40.094102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.094384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.094652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.118754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:40.120633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:40.125623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.142241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1009a-310b-4ed4-829b-87069c25a6b5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.151156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.180817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.181422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.182023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.182023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.182616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:40.184273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:40.197796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 137/278] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:40.198796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.217023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 142/279] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:15:40 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:15:40.319232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59108800-90e7-46ef-aec0-be69eb61819a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.362744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.363779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.364278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.364743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.366240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.387171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:40.395127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:40.396107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:40.396911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:40.399283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 138/280] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:40.405877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.438518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.499582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39409b76-306e-4aec-ae50-34109e60d012 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.524443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681a176-92d0-493d-bd04-7d2f455c626f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.537169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.537942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.538223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.538489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.558439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:40.564449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:40.565382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:40.566605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 143/281] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:40.567483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.570352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.571315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.571623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.571902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.572773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.573758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:40.575507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:40.576275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:40.588366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 139/282] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:40.589198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.693275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87abf-b86e-4495-a3b3-d7906c3ecd7a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.701818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.722777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde5955-87bb-4945-a67b-dd3c7706b158 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.730518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.731242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.731624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.732065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.743150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:40.749739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:40.751073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:40.751912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:40.755703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 144/283] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:40.756718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.763800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.764425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.764681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.764961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.765631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:40.765935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:40.767771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:40.780803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 140/284] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:40.782308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.855960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.893866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f21a0-49ae-41c8-a6d8-1d06a8696762 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.900799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095253be-dc77-4d5b-bdf4-7d5c645e4bec None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:40.933994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.934678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.934960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.935232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.935967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:40.936272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:40.937650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:40.947086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 141/285] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:40.947416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:40.948202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:40.948627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:40.948903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] PATH_INFO: `/v3/users/1f12c1af2e9549608c6433663740d74d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:40.952682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:40.969511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:40.972952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] RBAC: Authorizing `identity:get_user(user_id=1f12c1af2e9549608c6433663740d74d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:40.974736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:40.982044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 145/286] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/users/1f12c1af2e9549608c6433663740d74d => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:40.982044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:41.095418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3e2e9-a37e-4703-b3f6-7f1f8355bc82 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:41.121465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5284-c99b-4b8e-af99-350bb2ccceb9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:41.141179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:41.141924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:41.142207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:41.142578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] PATH_INFO: `/v3/users/46085f4526b241519ce46c9733f2755c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:41.143807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] RBAC: Authorizing `identity:get_user(user_id=46085f4526b241519ce46c9733f2755c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:41.145271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:41.147006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 142/287] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/users/46085f4526b241519ce46c9733f2755c => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:41.147806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:41.153976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:41.179308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:41.179308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:41.179308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:41.179741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] PATH_INFO: `/v3/users/119aab44bbdc4f93aa8a4a928122b980` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:41.182846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] RBAC: Authorizing `identity:get_user(user_id=119aab44bbdc4f93aa8a4a928122b980)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:41.184401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:41.186231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 146/288] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:40 2022] GET /identity/v3/users/119aab44bbdc4f93aa8a4a928122b980 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:41.190891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:41.271822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:41.302325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038462-f0ae-45d6-a6ca-593384fb4c2a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:41.319849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac3553-d68a-4df3-98b0-42b2c5eaa289 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:41.352761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:41.353935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:41.354872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:41.355265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:41.356123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, user.id=1f12c1af2e9549608c6433663740d74d, scope.system=all) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:41.356533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:41.358255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:41.367345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:41.367622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 143/289] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Sep 9 06:15:41 2022] GET /identity/v3/role_assignments?user.id=1f12c1af2e9549608c6433663740d74d&scope.system=all&role.id=660e237685d342e8bb6bef00916bca0b => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:41.368411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:41.368889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:41.369115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:41.369626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:41.371424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:41.373336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:41.375722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 147/290] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:41 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:41.390477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:41.534527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:41.613262 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec147f-0b3c-4c7b-a789-d66144658b38 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:41.648116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c1821-9a91-4e58-af98-9af8c125d6b0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:41.660818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:41.661449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:41.661729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:41.662003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:41.683851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:41.686370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:41.687050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:41.687438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:41.687746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649/users/46085f4526b241519ce46c9733f2755c/roles/73bcb9d47c704919b2f24af36135d40f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:41.693285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] RBAC: Authorizing `identity:create_grant(project_id=13c11b499ec641ce8a2a829378a48649, user_id=46085f4526b241519ce46c9733f2755c, role_id=73bcb9d47c704919b2f24af36135d40f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:41.693285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:41.737993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 148/291] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:15:41 2022] PUT /identity/v3/projects/13c11b499ec641ce8a2a829378a48649/users/46085f4526b241519ce46c9733f2755c/roles/73bcb9d47c704919b2f24af36135d40f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:41.771430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:41.773008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:41.897525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 144/292] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:41.906918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:42.075147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd299a56-e302-4b0b-9347-40197e12d432 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:42.096787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:42.119502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be84090-e3bc-46d8-9b38-8c32210f1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:42.120568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84090-e3bc-46d8-9b38-8c32210f1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:42.121070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84090-e3bc-46d8-9b38-8c32210f1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:42.121539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84090-e3bc-46d8-9b38-8c32210f1bd9 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:42.122934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be84090-e3bc-46d8-9b38-8c32210f1bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:42.124602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be84090-e3bc-46d8-9b38-8c32210f1bd9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:42.126730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 149/293] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:41 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:42.136292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:42.203088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:42.271042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76269128-8623-4880-9f0b-670539e68ee7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:42.312156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:42.312798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:42.313074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:42.313345 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:42.314076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=119aab44bbdc4f93aa8a4a928122b980) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:42.314381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:42.315835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2c22-7b41-4822-afe3-80d65fcab0c3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:42.323088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 145/294] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:42 2022] GET /identity/v3/role_assignments?user.id=119aab44bbdc4f93aa8a4a928122b980&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=660e237685d342e8bb6bef00916bca0b => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:43.000738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5bcb6-4f99-4282-849f-91a39235a08b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.001755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5bcb6-4f99-4282-849f-91a39235a08b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.002239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5bcb6-4f99-4282-849f-91a39235a08b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.003815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 150/295] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:43.009592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.010976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.011452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.015903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b563c6-2b32-4705-a405-287605f9e5be None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.016411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b563c6-2b32-4705-a405-287605f9e5be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.016905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b563c6-2b32-4705-a405-287605f9e5be None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.018636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 151/296] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:43.030746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.031078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.031291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.034515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:43.036086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:43.059140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:43.060355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:43.112888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 146/297] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:43.139870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 152/298] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:43.389025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:43.411498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:43.431425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:43.504876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b139a67-6c07-4847-8edb-b967d709758e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:43.538419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fe2aa-7e9c-4aa7-a064-371c13aedd9a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:43.548843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:43.549918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.550394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.550862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.552213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:43.554064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:43.556185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:43.557417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 147/299] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:43.561533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:43.586462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:43.587106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.587388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.587697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.601220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:43.607819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:43.608713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:43.609568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 153/300] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:43.617217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:43.681795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61349f7-d3ee-42ea-b97f-7249867ea133 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:43.684961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:43.726111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:43.727119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.727537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.727985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] PATH_INFO: `/v3/groups` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.728788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:43.730976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:43.731169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10188fc4-e45b-4505-b0d6-a669a9d39ac7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:43.749428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 148/301] 10.222.0.25 () {60 vars in 1134 bytes} [Fri Sep 9 06:15:43 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:15:43.776902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:43.777917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.778294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.778695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.779514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:43.779944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:43.781547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3146079-8bb7-490c-a6c1-09ac3b461acb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:43.792146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 154/302] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:43.798487 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:43.826637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def31a8-6007-494a-9e55-642b31d64cca None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.826925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def31a8-6007-494a-9e55-642b31d64cca None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.827199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def31a8-6007-494a-9e55-642b31d64cca None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.832965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 155/303] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:43.834550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.834834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.835107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:43.857585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:43.858810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:43.892613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:43.947657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 156/304] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:43.947861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b890d95-2205-4322-8797-2a2a4c58d849 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:43.985175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:43.985879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:43.986212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:43.986491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] PATH_INFO: `/v3/users/fd86cf8c6e574db7ac4f397d2c3370bf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.002052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] RBAC: Authorizing `identity:get_user(user_id=fd86cf8c6e574db7ac4f397d2c3370bf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:44.003513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:44.005486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 149/305] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:43 2022] GET /identity/v3/users/fd86cf8c6e574db7ac4f397d2c3370bf => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:44.011458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:44.129676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca13330-2be2-43a9-b689-4bfb71c8961f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:44.165939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:44.166552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.167358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.167667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.168509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] RBAC: Adding query filter params (role.id=73bcb9d47c704919b2f24af36135d40f, user.id=fd86cf8c6e574db7ac4f397d2c3370bf, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:44.168815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:44.170262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9ecc2-0dce-4925-986e-879ef1e597a9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:44.176023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 157/306] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Sep 9 06:15:44 2022] GET /identity/v3/role_assignments?user.id=fd86cf8c6e574db7ac4f397d2c3370bf&scope.system=all&role.id=73bcb9d47c704919b2f24af36135d40f => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:44.238653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:44.269185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:44.360794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3c38a-9559-4ba6-bc16-bded229d77fc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:44.372486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1a4f1-d31a-45a6-aae9-d4d3c04fa7c4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.372780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1a4f1-d31a-45a6-aae9-d4d3c04fa7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.373028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1a4f1-d31a-45a6-aae9-d4d3c04fa7c4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.374383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 158/307] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:44.381013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.381514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.382019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.402782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:44.403428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.403768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.404044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.414643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:44.419806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:44.420698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:44.421731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 150/308] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:44.426406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:44.428602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:44.430052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:44.459784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:44.518540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:44.521343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 159/309] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:44.540399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808ea9b-9742-473f-a025-af12f42412b5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:44.581285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:44.582039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.582325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.582597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.583269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:44.583599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:44.584996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:44.594856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 151/310] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:44.600942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:44.706046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e7dc-6471-4d05-8d9a-8ec00bacf2cd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:44.743900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:44.744548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.745413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.745692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] PATH_INFO: `/v3/users/1f12c1af2e9549608c6433663740d74d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.747085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] RBAC: Authorizing `identity:get_user(user_id=1f12c1af2e9549608c6433663740d74d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:44.748634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:44.750630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 160/311] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:44 2022] GET /identity/v3/users/1f12c1af2e9549608c6433663740d74d => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:44.756610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:44.810641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:44.849011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:44.878562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2d596-a7b4-4d09-9160-ff88ec1a956a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:44.917043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:44.917996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.918464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.918750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] PATH_INFO: `/v3/system/users/1f12c1af2e9549608c6433663740d74d/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:44.925755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f12c1af2e9549608c6433663740d74d, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:44.925755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:44.934670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 152/312] 10.222.0.25 () {58 vars in 1334 bytes} [Fri Sep 9 06:15:44 2022] PUT /identity/v3/system/users/1f12c1af2e9549608c6433663740d74d/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:44.944213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50515c4f-ad9a-4794-adde-df43cb4136d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:44.986479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:44.987468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:44.987917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:44.988336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:45.002616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:45.008088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:45.009247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:45.010809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:45.013131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 161/313] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:45.019622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:45.043948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:45.135127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659430b2-8e31-4151-8a4d-40ad82ab3d45 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:45.174348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:45.174972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:45.175254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:45.175527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:45.176220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:45.176527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:45.177897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:45.187994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 153/314] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:45.193368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:45.293565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:45.303118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9174586-c2a3-4653-8457-9becb26539be None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:45.341795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:45.342435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:45.342706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:45.342970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] PATH_INFO: `/v3/users/46085f4526b241519ce46c9733f2755c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:45.344142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] RBAC: Authorizing `identity:get_user(user_id=46085f4526b241519ce46c9733f2755c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:45.345577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:45.347510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 162/315] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:45 2022] GET /identity/v3/users/46085f4526b241519ce46c9733f2755c => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:45.352761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:45.427378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:45.460624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1596edd-b0fe-402b-88c4-852815972a49 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:45.496366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:45.496988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:45.497267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:45.497540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:45.573720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:45.575521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:45.648100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:45.680356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 154/316] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:45.686309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:45.792590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331760-8461-46c9-9eae-0c22662e8191 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:45.828203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc2ae5-6f06-4c86-8fb9-c17d51bab58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:45.829188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2ae5-6f06-4c86-8fb9-c17d51bab58a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:45.829667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2ae5-6f06-4c86-8fb9-c17d51bab58a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:45.830390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc2ae5-6f06-4c86-8fb9-c17d51bab58a None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:45.831740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc2ae5-6f06-4c86-8fb9-c17d51bab58a None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:45.833429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc2ae5-6f06-4c86-8fb9-c17d51bab58a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:45.835259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 163/317] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:45 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:45.841592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:45.952522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe6e4-e8d5-4880-8fa0-44cbccfb6002 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:45.990418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:45.991099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:45.991381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:45.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:45.992442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] RBAC: Adding query filter params (role.id=73bcb9d47c704919b2f24af36135d40f, scope.project.id=13c11b499ec641ce8a2a829378a48649, user.id=46085f4526b241519ce46c9733f2755c) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:45.992752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:45.994268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94857485-9e54-4a78-a294-549e7d250804 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:46.000991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 155/318] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:15:45 2022] GET /identity/v3/role_assignments?user.id=46085f4526b241519ce46c9733f2755c&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=73bcb9d47c704919b2f24af36135d40f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:46.142410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06371-33a7-4bde-a9c1-277a9b2b4ce0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:46.142931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06371-33a7-4bde-a9c1-277a9b2b4ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:46.142931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b06371-33a7-4bde-a9c1-277a9b2b4ce0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:46.144738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 164/319] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:46.150685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b3507-223e-4752-8161-2d41da8d96fc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:46.151507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b3507-223e-4752-8161-2d41da8d96fc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:46.151841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b3507-223e-4752-8161-2d41da8d96fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:46.177065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5e9b3507-223e-4752-8161-2d41da8d96fc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:46.178289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e9b3507-223e-4752-8161-2d41da8d96fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:46.206857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:46.250983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 156/320] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:46.498263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa1392-4681-460b-a912-d534ef0a2b4c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:46.498600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa1392-4681-460b-a912-d534ef0a2b4c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:46.498718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa1392-4681-460b-a912-d534ef0a2b4c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:46.500254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 165/321] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:46.505209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:46.505637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:46.505809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:46.524468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:46.525028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa1392-4681-460b-a912-d534ef0a2b4c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:46.525641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:46.555898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:46.607710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 157/322] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:46.637273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa1392-4681-460b-a912-d534ef0a2b4c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:46.673087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:46.674045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:46.674528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:46.675001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:46.676365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:46.677996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:46.679894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 166/323] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:15:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:46.685872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:46.781392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:46.788804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7ae24-5aec-4200-af9f-3bbaf636b7e1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:46.824189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:46.824808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:46.825087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:46.825356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] PATH_INFO: `/v3/groups` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:46.825984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:46.827404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:46.844681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 158/324] 10.222.0.25 () {60 vars in 1134 bytes} [Fri Sep 9 06:15:46 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:15:46.883517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:46.988605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:46.996898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a4467-3f81-406f-8a79-34a4b8e4290b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:47.037852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:47.038418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.038697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.038967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.049476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:47.054409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:47.055270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:47.056139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 167/325] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:47.062075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:47.171486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728489a-33da-408a-93a2-c7ad98a53b87 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:47.219227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:47.219912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.220191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.220463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.221136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:47.221438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:47.222803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef437-8180-4772-90cb-770227be16c2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:47.233139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 159/326] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:47.238306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:47.312563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2af89e-7241-4698-a6e6-2d09aa934cd9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.312816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2af89e-7241-4698-a6e6-2d09aa934cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.313051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2af89e-7241-4698-a6e6-2d09aa934cd9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.314686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 160/327] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:47.320071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.321057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.321057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.343230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:47.344505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:47.356380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9beb55-3357-465f-a8c6-5bd6b85cd225 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:47.358471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:47.417144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:47.417910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.418188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.418458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] PATH_INFO: `/v3/users/fd86cf8c6e574db7ac4f397d2c3370bf` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.419754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] RBAC: Authorizing `identity:get_user(user_id=fd86cf8c6e574db7ac4f397d2c3370bf)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:47.421121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:47.423008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 168/328] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:47 2022] GET /identity/v3/users/fd86cf8c6e574db7ac4f397d2c3370bf => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:47.428209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:47.443961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 161/329] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:47.459983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:47.577338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a23df-2972-4676-86a2-3b5e6c139dbe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:47.621991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:47.622651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.622926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.623198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] PATH_INFO: `/v3/system/users/fd86cf8c6e574db7ac4f397d2c3370bf/roles/73bcb9d47c704919b2f24af36135d40f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.624635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fd86cf8c6e574db7ac4f397d2c3370bf, role_id=73bcb9d47c704919b2f24af36135d40f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:47.626013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:47.638256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 169/330] 10.222.0.25 () {58 vars in 1334 bytes} [Fri Sep 9 06:15:47 2022] PUT /identity/v3/system/users/fd86cf8c6e574db7ac4f397d2c3370bf/roles/73bcb9d47c704919b2f24af36135d40f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:47.781988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:47.897953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e804d-7130-4484-8778-f5a191ddd12d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:47.937939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:47.938955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:47.939434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:47.939933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:47.953036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:47.958793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:47.959725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:47.960563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:47.964785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 162/331] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:47.972745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:48.100058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:48.119270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869012d-6584-4959-a687-bf4ff4ad0efc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:48.169412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:48.169964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:48.170239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:48.170506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:48.171177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:48.171478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:48.172886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:48.182234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 170/332] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:48.188015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:48.204170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:48.328775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e059f34-04c4-45fc-809c-83bc17e2748f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:48.365568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e4854-99c4-4ebe-bc5d-6aae03ba4dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:48.366213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e4854-99c4-4ebe-bc5d-6aae03ba4dad None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:48.366492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e4854-99c4-4ebe-bc5d-6aae03ba4dad None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:48.366764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e4854-99c4-4ebe-bc5d-6aae03ba4dad None admin] PATH_INFO: `/v3/users/1f12c1af2e9549608c6433663740d74d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:48.367933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e4854-99c4-4ebe-bc5d-6aae03ba4dad None admin] RBAC: Authorizing `identity:get_user(user_id=1f12c1af2e9549608c6433663740d74d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:48.369369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e4854-99c4-4ebe-bc5d-6aae03ba4dad None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:48.371484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 163/333] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:48 2022] GET /identity/v3/users/1f12c1af2e9549608c6433663740d74d => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:48.377575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:48.504813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1be65-9ff0-48dd-bf90-73e9ce1b7f34 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:48.551403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:48.552052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:48.552328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:48.552662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:48.553403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] RBAC: Adding query filter params (role.id=660e237685d342e8bb6bef00916bca0b, user.id=1f12c1af2e9549608c6433663740d74d, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:48.553710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:48.555141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f78fb-3ad5-4477-900a-20d96f8b569a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:48.572185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 171/334] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Sep 9 06:15:48 2022] GET /identity/v3/role_assignments?user.id=1f12c1af2e9549608c6433663740d74d&scope.system=all&role.id=660e237685d342e8bb6bef00916bca0b => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:50.267207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68aca9-b945-4d39-833d-4e3b6607eaf7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.267762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68aca9-b945-4d39-833d-4e3b6607eaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.267762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68aca9-b945-4d39-833d-4e3b6607eaf7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.269101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 164/335] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:50.281842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8109-9af3-45ea-a76d-7c851372c28d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.282347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8109-9af3-45ea-a76d-7c851372c28d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.282424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8109-9af3-45ea-a76d-7c851372c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.299015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:50.305176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-991d8109-9af3-45ea-a76d-7c851372c28d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:50.306367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-991d8109-9af3-45ea-a76d-7c851372c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:50.337330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20c00c-1849-4c64-8a03-60242ff18373 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.337620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20c00c-1849-4c64-8a03-60242ff18373 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.338019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20c00c-1849-4c64-8a03-60242ff18373 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.340260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 165/336] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:50.346158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871343-7932-4cf3-ad41-2e8dda03530e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.346503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871343-7932-4cf3-ad41-2e8dda03530e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.346844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871343-7932-4cf3-ad41-2e8dda03530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.374321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f9871343-7932-4cf3-ad41-2e8dda03530e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:50.375524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f9871343-7932-4cf3-ad41-2e8dda03530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:50.425242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 172/337] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:50.486950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 166/338] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:50.648062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b174-134b-4ef5-9da1-b9e97ec1a538 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.648343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b174-134b-4ef5-9da1-b9e97ec1a538 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.648775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b174-134b-4ef5-9da1-b9e97ec1a538 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.650273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 173/339] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:50.656281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a90713-6aa6-4a5a-b426-a20b303ac752 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.656836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a90713-6aa6-4a5a-b426-a20b303ac752 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.657256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a90713-6aa6-4a5a-b426-a20b303ac752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.666007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:50.680500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-98a90713-6aa6-4a5a-b426-a20b303ac752 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:50.681698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-98a90713-6aa6-4a5a-b426-a20b303ac752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:50.771193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 167/340] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:50.797839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6faa-bd73-4cbb-a058-a56f25f280de None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.798128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6faa-bd73-4cbb-a058-a56f25f280de None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.798312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6faa-bd73-4cbb-a058-a56f25f280de None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.799682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 174/341] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:50.810070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:50.810340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:50.810613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:50.844610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:50.845907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:50.887612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc6faa-bd73-4cbb-a058-a56f25f280de None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:50.977918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 168/342] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:51.057823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc6faa-bd73-4cbb-a058-a56f25f280de None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.103069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.104296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.104765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.105182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.112736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:51.120320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:51.124886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:51.127828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:51.128739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:51.129690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 175/343] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:51.134696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.252082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:51.282285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9466b2c-02b4-4142-93ee-2b45b15cfc2a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.295905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ac87c-6f05-4bb9-b76c-cf7f61208840 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.343818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.344822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.345366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.345774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.358416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:51.364281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:51.365199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:51.366000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:51.368537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 169/344] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:51.370223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.370716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.371409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.371798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.372131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.373093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:51.373464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:51.380974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:51.401577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 176/345] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:51.402716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.485632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82269d2e-6e90-431e-9376-3d637ca257d9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.498472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:51.512643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:51.523919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.526662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.526662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.526662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.541766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:51.551045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:51.552024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 06:15:51.552853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:51.555585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 170/346] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:51.557662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.573110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cb515-3e7f-4e2f-8636-8c872c050b60 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.617763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.618435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.618781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.619756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.621017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:51.621388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:51.622950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:51.634001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 177/347] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:51.635466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.689547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074319d3-be0b-4e88-b53d-a933bb93078e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.734913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.735946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.736464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.736960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.750981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:51.756746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:51.757723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:51.758544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:51.764056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 171/348] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:51.765954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:51.780791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b501abe-e488-4753-b137-9a596293bcdf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.809550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:51.825143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:51.834419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.834626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.834893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.835127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.899079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f851fd2-458e-4b6a-b09f-15ddeaf796fe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:51.938360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:51.940469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:51.956800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:51.957427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:51.957711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:51.957985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:51.958677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:51.958985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:51.960409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:51.969440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 172/349] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:51.970295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.065078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 178/350] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:52.066006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.092069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.094344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977aba-277f-4d22-ac97-7c94991900ad None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:52.103471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.138880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c719a-f4e6-44a2-ace6-322e645b822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:52.139685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c719a-f4e6-44a2-ace6-322e645b822e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.139968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c719a-f4e6-44a2-ace6-322e645b822e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.140240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c719a-f4e6-44a2-ace6-322e645b822e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.185465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0c578-77b9-4911-860c-9f2eb8257eba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:52.224924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:52.225841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.226118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.226233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c719a-f4e6-44a2-ace6-322e645b822e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:52.226403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.227057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:52.227354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:52.227609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c719a-f4e6-44a2-ace6-322e645b822e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:52.228972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f0e16-90b1-44df-8604-ffccaa833ea0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:52.239489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 179/351] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:52.240779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf49e6c-883a-4678-8ec4-18ce3c731539 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.241185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf49e6c-883a-4678-8ec4-18ce3c731539 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.241457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf49e6c-883a-4678-8ec4-18ce3c731539 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.242780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 180/352] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:52.243338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf49e6c-883a-4678-8ec4-18ce3c731539 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.349440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.356267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 173/353] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:52.359585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9098b7-8faa-4207-9b07-5e7a8f819517 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.359585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9098b7-8faa-4207-9b07-5e7a8f819517 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.359585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9098b7-8faa-4207-9b07-5e7a8f819517 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.359585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 174/354] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:52.359940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9098b7-8faa-4207-9b07-5e7a8f819517 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.370455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.382310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf49e6c-883a-4678-8ec4-18ce3c731539 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:52.438471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:52.438735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.439018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.439284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] PATH_INFO: `/v3/users/fd86cf8c6e574db7ac4f397d2c3370bf` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.441249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] RBAC: Authorizing `identity:get_user(user_id=fd86cf8c6e574db7ac4f397d2c3370bf)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:52.442692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:52.444474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 181/355] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/users/fd86cf8c6e574db7ac4f397d2c3370bf => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:52.449082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.488769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9098b7-8faa-4207-9b07-5e7a8f819517 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:52.533846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd793a9-15bf-4c62-8469-b1492475827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:52.534949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd793a9-15bf-4c62-8469-b1492475827e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.535387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd793a9-15bf-4c62-8469-b1492475827e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.535698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd793a9-15bf-4c62-8469-b1492475827e None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.537691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd793a9-15bf-4c62-8469-b1492475827e None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:52.539145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd793a9-15bf-4c62-8469-b1492475827e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:52.542867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 175/356] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:52.544856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.545140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.545416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.570244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:52.570585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849950-57ba-4570-955c-49e9cc04aae9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:52.570812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:52.608714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5254223-1cb4-4f57-8905-fb613ea13be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:52.609343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5254223-1cb4-4f57-8905-fb613ea13be1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.609623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5254223-1cb4-4f57-8905-fb613ea13be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.609897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5254223-1cb4-4f57-8905-fb613ea13be1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.638384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.654801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 176/357] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:52.659864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.665038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.695258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5254223-1cb4-4f57-8905-fb613ea13be1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:52.696677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5254223-1cb4-4f57-8905-fb613ea13be1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:52.802851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810a0b-0f3f-466a-9d5c-2001ea39f2ee None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:52.822671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 182/358] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:52.822671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.822671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.822671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.841456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:52.842667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:52.844103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:52.844750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:52.845026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:52.845296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:52.847086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:52.848559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:52.850193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 177/359] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:52.850969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.895487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.926595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 183/360] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:52.927994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:52.935943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:52.968432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b04fe-4ea1-4fc4-bbc5-1c2270161052 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.012503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.012503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.012503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.012861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.013564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] RBAC: Adding query filter params (role.id=73bcb9d47c704919b2f24af36135d40f, user.id=fd86cf8c6e574db7ac4f397d2c3370bf, scope.system=all) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:53.013875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.015307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.031589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 178/361] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/role_assignments?user.id=fd86cf8c6e574db7ac4f397d2c3370bf&scope.system=all&role.id=73bcb9d47c704919b2f24af36135d40f => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.032420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.058724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322974d-20af-47d2-bb9b-4d4a99b6547c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.106635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.107302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.107610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.107892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.117137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.120025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.122911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 184/362] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:52 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.124414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.178879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8ec6a-5799-41e7-86c2-4b95c9d004f1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.197912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:53.219596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:53.233463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.234128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.234409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.234694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.235993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.237511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.239231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 179/363] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.240168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.273893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6f49f-c63d-49fa-8962-f06cc8ec4af8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.316101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.317124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.317827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.318331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.320313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.323090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.327460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 185/364] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.328533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.373045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5b115-0807-4426-a70a-f234e85873dc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.414051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.415279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.415279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.415279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.426524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:53.432706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:53.433648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:53.434495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 180/365] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:53.435159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.459923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8b6c9-5cfb-4772-8b86-cd4e3f52837b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.476250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:53.500441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.501060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.501341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.501614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.502355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=127332367fa24e318828c23b1f6090cc) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:53.502664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.504440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.506146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:53.513262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 186/366] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=660e237685d342e8bb6bef00916bca0b => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.513536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.573829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519902c-b78f-4b3e-ae2a-3e17c2e1eeb9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.627182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.630489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.630489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.630489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.643128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:53.648774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:53.649734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:53.651073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 181/367] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:53.651930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.661277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6c98-58c6-406e-a91f-3f64b3ce63a8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.705284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.705932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.706214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.706487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.708264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.709729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.712381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 187/368] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.713507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.768099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:53.795392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262fcf5-68c3-4146-81b6-9278957d382e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.817450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:53.840861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.841475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.841914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.842193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.843498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=13c11b499ec641ce8a2a829378a48649) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:53.843847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.845287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.845381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615fbeb-99c1-4bc1-aa46-9d857967d9c4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:53.859575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 182/369] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=660e237685d342e8bb6bef00916bca0b => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.859575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.893659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:53.894449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:53.894728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:53.895052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:53.895802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:53.896123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:53.897602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:53.911634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 188/370] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:53.914643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:53.992151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf3ad1-3df7-4c93-b3eb-aec00998a2ae None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.039761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.040310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.040596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.040871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.042080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:54.042385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.044287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.047732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af362868-a6a6-4cc3-aeaf-2f89610aef0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.055480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:54.057645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 183/371] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:54.061740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.094177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.095061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.095342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.095646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.096540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=127332367fa24e318828c23b1f6090cc) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:54.096854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.098911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.106718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 189/372] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:15:53 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:54.107571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.134489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:54.239214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa79bc-90ec-40e0-a206-d0b15c2b7b55 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.300459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.300981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.301311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.301586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.317011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20db52-9785-499a-ab30-370c38942f08 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.358170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.358790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.359070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.359344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.411994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:54.442377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.443873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.489636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.491131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.554883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 190/373] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:54.561481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.606483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:54.664116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:54.699819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187373-2447-41bf-a07c-9f473ffd498e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.711721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 184/374] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:15:54.717409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.748254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.748908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.749289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.749570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] PATH_INFO: `/v3/projects/df82cd1e5393473c92978bf22c792e3e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.756370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] RBAC: Authorizing `identity:get_project(project_id=df82cd1e5393473c92978bf22c792e3e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.757848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.761051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 191/375] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/projects/df82cd1e5393473c92978bf22c792e3e => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:54.770982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.878372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690133-4da1-43b7-8574-c1b8ab7b7cac None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.887399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c0cdf-9d6b-48d1-a2b1-165b10da1af8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:54.924423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.925063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.925342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.925714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] PATH_INFO: `/v3/groups/ea422491970a4b37a9cd16b546561f73` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.933200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] RBAC: Authorizing `identity:get_group(group_id=ea422491970a4b37a9cd16b546561f73)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.933840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.936720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 192/376] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/groups/ea422491970a4b37a9cd16b546561f73 => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:54.940638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:54.941289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:54.941569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:54.942627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.943048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:54.948070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:54.948442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:54.949923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:54.952123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 185/377] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:54.957918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:54.990192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:55.055469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948cc57-c877-4d01-be82-569682de5a3e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:55.094357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:55.095317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:55.095824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:55.096293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:55.097210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] RBAC: Adding query filter params (group.id=ea422491970a4b37a9cd16b546561f73, role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=df82cd1e5393473c92978bf22c792e3e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:55.097708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:55.099315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:55.106084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 193/378] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/role_assignments?group.id=ea422491970a4b37a9cd16b546561f73&scope.project.id=df82cd1e5393473c92978bf22c792e3e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:55.190531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb2bb8-e138-4a7d-b56b-60a1fd32c2ff None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:55.239098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41c94f-4a95-4d45-aa9b-da4dbb38a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:55.240109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41c94f-4a95-4d45-aa9b-da4dbb38a5ce None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:55.240585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41c94f-4a95-4d45-aa9b-da4dbb38a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:55.241059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41c94f-4a95-4d45-aa9b-da4dbb38a5ce None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:55.245279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41c94f-4a95-4d45-aa9b-da4dbb38a5ce None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:55.247005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41c94f-4a95-4d45-aa9b-da4dbb38a5ce None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:55.248893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 186/379] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:54 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:55.255119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:55.336969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:55.376337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8aa0b-262a-4b6c-896a-8019201581c3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:55.421216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:55.422153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:55.422621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:55.423085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:55.424024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=13c11b499ec641ce8a2a829378a48649) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:55.424513 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:55.426140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc72cd1-cf6a-4a21-953a-c10a9dc7f263 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:55.432748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 194/380] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:15:55 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:56.063339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4507b-1206-4bda-b1d7-d3f6f0510fb7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.064380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4507b-1206-4bda-b1d7-d3f6f0510fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.064875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4507b-1206-4bda-b1d7-d3f6f0510fb7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.066717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 187/381] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:56.073562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.074430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.074907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.095901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:56.097479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:56.169696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 195/382] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:56.455193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4507b-1206-4bda-b1d7-d3f6f0510fb7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:56.553690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:56.584472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4507b-1206-4bda-b1d7-d3f6f0510fb7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:56.635677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b649e4-23ba-426d-98a3-33bdf2c9863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:56.636321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b649e4-23ba-426d-98a3-33bdf2c9863f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.636600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b649e4-23ba-426d-98a3-33bdf2c9863f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.636868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b649e4-23ba-426d-98a3-33bdf2c9863f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.652575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b649e4-23ba-426d-98a3-33bdf2c9863f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e4b649e4-23ba-426d-98a3-33bdf2c9863f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:56.657624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:56.658544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:56.659385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 188/383] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:56.664801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:56.734400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:56.768304 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda58de2-a659-410b-80ee-b335f43ef934 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.768634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda58de2-a659-410b-80ee-b335f43ef934 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.768851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda58de2-a659-410b-80ee-b335f43ef934 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.770440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 189/384] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:15:56.776755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23548f8-f7d0-4e3f-9cd8-0f030522872a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.777120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23548f8-f7d0-4e3f-9cd8-0f030522872a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.777396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23548f8-f7d0-4e3f-9cd8-0f030522872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.803109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd12b3-9c76-4a39-bab3-e7cac4af70b4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:56.810652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a23548f8-f7d0-4e3f-9cd8-0f030522872a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:56.811881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a23548f8-f7d0-4e3f-9cd8-0f030522872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:56.849214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:56.850705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.850705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.850705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.851219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:56.851571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:56.855567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:56.863325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 196/385] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:56.870105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:56.931666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 190/386] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:56.933574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfc80a-6a04-46f2-a330-58df93e5a1af None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.933856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfc80a-6a04-46f2-a330-58df93e5a1af None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.934127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfc80a-6a04-46f2-a330-58df93e5a1af None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.936547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 191/387] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:56.953274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:56.953555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:56.953826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:56.987737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:56.988850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:57.038045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76948-1aea-4992-8963-f91108706715 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.090398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:57.098866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:57.106617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 192/388] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:15:57.122597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.123243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.123521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.123819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.128218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:57.129669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:57.131371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 197/389] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:56 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:57.138478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.214929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.295634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86758514-11a9-4436-bb8d-3a21e6fee8dc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.335473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.337010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.337505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.339140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.340614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:57.342400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:57.344944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 193/390] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:57.351313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.415604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63289e4a-0191-478a-99e2-dc851aafb447 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.436697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:57.464492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.465376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.465660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.466342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.475034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b43bd-5b76-4921-abea-1f3dce8710fa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.480239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:57.487128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:57.488130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:57.488981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 198/391] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:57.489659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.519001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.519755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.520034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.520309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc/groups/c25a751122844af9bff9e5a68900901a/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.522275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] RBAC: Authorizing `identity:create_grant(project_id=127332367fa24e318828c23b1f6090cc, group_id=c25a751122844af9bff9e5a68900901a, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:57.523797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:57.541722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 194/392] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Sep 9 06:15:57 2022] PUT /identity/v3/projects/127332367fa24e318828c23b1f6090cc/groups/c25a751122844af9bff9e5a68900901a/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:15:57.543416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.554592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:57.667188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefefb6-c031-4fa8-b39f-01dc16edd2a2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.678833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e914c0a-0521-4d75-9b6e-c3b26f5e0552 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.718364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:57.731402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.732521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.732805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.733077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.733762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:57.734062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:57.735468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:57.751663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 195/393] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:57.753134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.753350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.753942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.754224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.754497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.778302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:57.783997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:57.784941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:57.785763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:15:57.788883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 199/394] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:57.795503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.889219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b8500-1a63-474d-8cb2-adb3bc5167d2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.932684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c21d6e-32bc-4d37-b68c-719233d5a97e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:57.933581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3250-f712-43f7-8995-c884412681ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.934226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3250-f712-43f7-8995-c884412681ea None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.934507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3250-f712-43f7-8995-c884412681ea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.935051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3250-f712-43f7-8995-c884412681ea None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.937124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed3250-f712-43f7-8995-c884412681ea None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:57.938572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed3250-f712-43f7-8995-c884412681ea None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:57.943987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 196/395] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:57.943987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af704bb-a99d-4e64-849e-3f94d8b5e128 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.943987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af704bb-a99d-4e64-849e-3f94d8b5e128 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.943987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af704bb-a99d-4e64-849e-3f94d8b5e128 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.943987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 197/396] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:57.949193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af704bb-a99d-4e64-849e-3f94d8b5e128 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:57.952549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:57.982227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:57.983386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:57.984338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:57.985046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:57.985915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:57.986368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:57.988016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f798adf-7439-4ee4-8e35-501e90d391d0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:58.002053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 200/397] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:58.004634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476068a-76b1-4fe2-adce-6758023b0642 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.005381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476068a-76b1-4fe2-adce-6758023b0642 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.005831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476068a-76b1-4fe2-adce-6758023b0642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.018612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:58.033904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d476068a-76b1-4fe2-adce-6758023b0642 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:58.035372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d476068a-76b1-4fe2-adce-6758023b0642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:58.066439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af704bb-a99d-4e64-849e-3f94d8b5e128 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:58.110087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:58.110776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.110993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.111267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.112404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:58.113854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:58.121271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 198/398] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:57 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:58.121271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.132185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 201/399] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:58.133470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac430743-7ab3-48cb-9a37-3a909987d2d6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.133818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac430743-7ab3-48cb-9a37-3a909987d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.134155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac430743-7ab3-48cb-9a37-3a909987d2d6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.136169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 202/400] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:15:58.140467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac430743-7ab3-48cb-9a37-3a909987d2d6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.246040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c4179-770e-4dc9-aa88-95b424c47082 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:58.258805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac430743-7ab3-48cb-9a37-3a909987d2d6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:58.260697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:58.277460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:58.287659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca04e4b-ddb6-4f8f-b5b3-b808d1027162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:58.288744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca04e4b-ddb6-4f8f-b5b3-b808d1027162 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.289227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca04e4b-ddb6-4f8f-b5b3-b808d1027162 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.289726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca04e4b-ddb6-4f8f-b5b3-b808d1027162 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.291352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca04e4b-ddb6-4f8f-b5b3-b808d1027162 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:58.293147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca04e4b-ddb6-4f8f-b5b3-b808d1027162 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:58.295118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 199/401] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:58 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:58.296785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.296893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:58.297436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.297560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.297827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.298106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649/groups/c25a751122844af9bff9e5a68900901a/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.298438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.300235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=13c11b499ec641ce8a2a829378a48649, group_id=c25a751122844af9bff9e5a68900901a, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:58.301670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:58.319404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:15:58.320605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:15:58.330701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 203/402] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Sep 9 06:15:58 2022] PUT /identity/v3/projects/13c11b499ec641ce8a2a829378a48649/groups/c25a751122844af9bff9e5a68900901a/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:58.334727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.441186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 200/403] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:15:58.452925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.546094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6a272-1039-4f5f-8849-5d57894963f7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:58.595722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:58.596378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.596979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.597259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.598649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:58.600131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:58.602233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 204/404] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:58 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:58.608769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.665250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:58.680130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d02e87-5415-4063-812f-232359ffb929 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:58.729610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:58.748838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:58.749477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.749756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.750026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.764045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2ad7f-e2a0-45b0-ae8a-7623ebc12703 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:58.776647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:58.782017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:58.782910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:58.783786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 201/405] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:15:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:15:58.784465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.804155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:58.805075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:58.805741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:58.806072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc/groups/c25a751122844af9bff9e5a68900901a/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:58.809587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] RBAC: Authorizing `identity:create_grant(project_id=127332367fa24e318828c23b1f6090cc, group_id=c25a751122844af9bff9e5a68900901a, role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:58.811338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:58.831953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 205/406] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Sep 9 06:15:58 2022] PUT /identity/v3/projects/127332367fa24e318828c23b1f6090cc/groups/c25a751122844af9bff9e5a68900901a/roles/8ace72a8232f495d8e03de411d3ec43b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:58.832786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:58.951574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ff121-75b7-4e0a-b05e-57ddface1ce2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.020257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:59.031531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.032215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.032497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.032773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.041183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cfbbc-ea1a-45ce-801e-49f24702efee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.051253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:15:59.057282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:15:59.058201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:15:59.059027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 202/407] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:15:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:15:59.064660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.100671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.101288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.101572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.101842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.102512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:59.102817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.104483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.117444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 206/408] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:15:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:59.123566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.128749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:59.222293 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559296cf-b414-467d-ba22-5885a6a130c4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.270827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.271658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.271943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.272220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.273310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:15:59.273626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.275417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.289527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 203/409] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:15:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:59.295636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.304963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260732ef-511b-4321-9b97-b08508a1b8df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.360741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.361883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.363855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.365719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] PATH_INFO: `/v3/groups/ea422491970a4b37a9cd16b546561f73` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.367055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] RBAC: Authorizing `identity:get_group(group_id=ea422491970a4b37a9cd16b546561f73)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.370111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.374502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 207/410] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:59 2022] GET /identity/v3/groups/ea422491970a4b37a9cd16b546561f73 => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:59.382018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.427300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:59.505676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117f780-4daf-4264-b010-4fbdcc2e33df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.520665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:59.529539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504238e2-cae4-4049-804e-15d69161ad01 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.584885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.585538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.585818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.586093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.587990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.590085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.592022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.592716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.592997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.593267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] PATH_INFO: `/v3/projects/df82cd1e5393473c92978bf22c792e3e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.594823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] RBAC: Authorizing `identity:get_project(project_id=df82cd1e5393473c92978bf22c792e3e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.596289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.598265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 204/411] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:15:59 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:59.598728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 208/412] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:59 2022] GET /identity/v3/projects/df82cd1e5393473c92978bf22c792e3e => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:15:59.599616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.604248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.776415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:15:59.794525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d4d2-2b1f-4a5b-babf-f2d7cf1e2840 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.797738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ad8f0-08f1-45a4-b668-a13af7d35ddd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:15:59.855667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d68ab-219f-40e9-a0fa-80a8c8f33283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.856346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d68ab-219f-40e9-a0fa-80a8c8f33283 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.856629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d68ab-219f-40e9-a0fa-80a8c8f33283 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.856903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d68ab-219f-40e9-a0fa-80a8c8f33283 None admin] PATH_INFO: `/v3/projects/df82cd1e5393473c92978bf22c792e3e/groups/ea422491970a4b37a9cd16b546561f73/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.860538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d68ab-219f-40e9-a0fa-80a8c8f33283 None admin] RBAC: Authorizing `identity:create_grant(project_id=df82cd1e5393473c92978bf22c792e3e, group_id=ea422491970a4b37a9cd16b546561f73, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.862019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d68ab-219f-40e9-a0fa-80a8c8f33283 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.865133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:15:59.865789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:15:59.866067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:15:59.866339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:15:59.867607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:15:59.869772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:15:59.872915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 205/413] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:15:59 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:15:59.880699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:15:59.894895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 209/414] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Sep 9 06:15:59 2022] PUT /identity/v3/projects/df82cd1e5393473c92978bf22c792e3e/groups/ea422491970a4b37a9cd16b546561f73/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:15:59.932401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:00.009235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422802c-8c70-48a7-9c8c-b43fd88d159f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:00.058316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e9b63-796f-47a8-846b-b18149aa05c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:00.058926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e9b63-796f-47a8-846b-b18149aa05c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:00.059210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e9b63-796f-47a8-846b-b18149aa05c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:00.059488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e9b63-796f-47a8-846b-b18149aa05c2 None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649/groups/c25a751122844af9bff9e5a68900901a/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:00.061758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e9b63-796f-47a8-846b-b18149aa05c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=13c11b499ec641ce8a2a829378a48649, group_id=c25a751122844af9bff9e5a68900901a, role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:00.063223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e9b63-796f-47a8-846b-b18149aa05c2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:00.091916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 206/415] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Sep 9 06:15:59 2022] PUT /identity/v3/projects/13c11b499ec641ce8a2a829378a48649/groups/c25a751122844af9bff9e5a68900901a/roles/8ace72a8232f495d8e03de411d3ec43b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:00.251945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63a70-7f3c-4b09-b102-70dc07b0eeba None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:00.252314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63a70-7f3c-4b09-b102-70dc07b0eeba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:00.252498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b63a70-7f3c-4b09-b102-70dc07b0eeba None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:00.253906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 210/416] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:00.261031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:00.261673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:00.261673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:00.281645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:00.282890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:00.358925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 207/417] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:00.648159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b63a70-7f3c-4b09-b102-70dc07b0eeba None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:00.787204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b63a70-7f3c-4b09-b102-70dc07b0eeba None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:00.832614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:00.833253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:00.833530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:00.833860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:00.850978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:00.857138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:00.858234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:00.859256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 211/418] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:00.865452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:00.913796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:00.989541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd43d-5aa1-4652-bb28-f7b9f46ca72d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:01.030294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:01.030918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.031207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.031488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.032196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:01.032507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:01.033896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48463e1f-19ca-4717-80e2-fbf377b5c258 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:01.042597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 208/419] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:16:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:01.048124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:01.067791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:01.111947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92147390-594e-48ca-9a52-d226ae513c2f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.112265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92147390-594e-48ca-9a52-d226ae513c2f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.112510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92147390-594e-48ca-9a52-d226ae513c2f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.114260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 209/420] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:01.120373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d231e56-4f9e-4453-906d-396e6c8a3dda None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.120938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d231e56-4f9e-4453-906d-396e6c8a3dda None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.121412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d231e56-4f9e-4453-906d-396e6c8a3dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.142106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9d231e56-4f9e-4453-906d-396e6c8a3dda None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:01.144727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d231e56-4f9e-4453-906d-396e6c8a3dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:01.182150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672c411-2f10-4b71-9521-fdd1ef3478bf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:01.240795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:01.241417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.241697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.241969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.250718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 210/421] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:01.350968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:01.352467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:01.365247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:01.375389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55bf3d-81d4-41e3-afae-fa495d228055 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.375910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55bf3d-81d4-41e3-afae-fa495d228055 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.376031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55bf3d-81d4-41e3-afae-fa495d228055 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.377631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 211/422] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:01.382539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.382827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.383103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.412323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:01.413553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:01.451185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:01.531672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 212/423] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:01.534703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:01.573622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 212/424] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:01.604929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:01.685159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060f84f-789b-4575-bd3c-cdb648a11316 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:01.695003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:01.722990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:01.723558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.723857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.724129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.725238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:01.726666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:01.727331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e961ff1-883b-4152-9279-f2ed4ddbc2b9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:01.734535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 213/425] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:16:01 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:01.735767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:01.780165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:01.782455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.782950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.784510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.811630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:01.817011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:01.817925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 06:16:01.818741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:01.823603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 213/426] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:01.839698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:01.863891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:01.907092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2090ae7-d915-431e-9f20-f9713672a022 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:01.943754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:01.944615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:01.945069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:01.945489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:01.947050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:01.948761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:01.950797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 214/427] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:16:01 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:01.956650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.007826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade1799-a767-4f9e-adf6-a3d1bffd9089 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.048143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.048759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.049041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.049317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.050136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:02.050280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:02.051862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:02.054953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:02.068209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 214/428] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:16:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:02.069567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.123910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcb6b8-93d6-4cdd-8104-702ae5c523cc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.168389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.169144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:02.169519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.170315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.170741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.171962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=127332367fa24e318828c23b1f6090cc) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:02.172581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:02.174346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:02.182859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 215/429] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:16:01 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=660e237685d342e8bb6bef00916bca0b => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:02.184453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.198335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8e913-601d-4359-9eed-26cfafe474ed None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.269226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.269812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.270093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.270365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.295560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:02.301642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:02.302613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:16:02.303509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:02.304309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 215/430] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:02.315535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.406660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba30803-317f-4cd7-911c-6905cb17ec44 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.419218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:02.455913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeefa6b-277e-49e2-b335-5d9fc522471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.456898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeefa6b-277e-49e2-b335-5d9fc522471c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.457436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeefa6b-277e-49e2-b335-5d9fc522471c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.457888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeefa6b-277e-49e2-b335-5d9fc522471c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.494086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99121-c88d-493c-a24d-ef42ced1f33c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.559680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.560724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.561187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.561666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.562545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:02.563030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:02.563145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeefa6b-277e-49e2-b335-5d9fc522471c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:02.564688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeefa6b-277e-49e2-b335-5d9fc522471c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:02.565013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:02.569161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:02.579116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 216/431] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:02.586259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.692584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 216/432] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:02.694004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfbfa2-cbe9-4015-b5be-c0811e942470 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.694261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfbfa2-cbe9-4015-b5be-c0811e942470 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.694533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfbfa2-cbe9-4015-b5be-c0811e942470 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.700475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 217/433] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:02.701352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfbfa2-cbe9-4015-b5be-c0811e942470 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.716404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:02.719808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a8ad7-c7a9-499f-93b1-73e165a171d9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.772557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc9b6c-596d-430c-86d7-1d6b169ef1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.772557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc9b6c-596d-430c-86d7-1d6b169ef1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.772557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc9b6c-596d-430c-86d7-1d6b169ef1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.772557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc9b6c-596d-430c-86d7-1d6b169ef1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.837102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfbfa2-cbe9-4015-b5be-c0811e942470 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:02.855326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:02.859231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc9b6c-596d-430c-86d7-1d6b169ef1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:02.860795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc9b6c-596d-430c-86d7-1d6b169ef1b7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:02.876149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83f719-98c2-439a-92ab-95c3e2c95e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:02.876784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83f719-98c2-439a-92ab-95c3e2c95e9e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.877063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83f719-98c2-439a-92ab-95c3e2c95e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.877335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83f719-98c2-439a-92ab-95c3e2c95e9e None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.878451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83f719-98c2-439a-92ab-95c3e2c95e9e None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:02.880903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83f719-98c2-439a-92ab-95c3e2c95e9e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:02.882526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 218/434] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:02.883785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.884030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.884343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.902668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:02.903912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:02.962834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 217/435] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:02.965006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510b49-7a7c-4357-87a9-c9511dc10a1e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:02.965302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510b49-7a7c-4357-87a9-c9511dc10a1e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:02.965580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510b49-7a7c-4357-87a9-c9511dc10a1e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:02.967053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 218/436] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:02.968017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0510b49-7a7c-4357-87a9-c9511dc10a1e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.974272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 219/437] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:02.975105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:02.983607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.069167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.075074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0510b49-7a7c-4357-87a9-c9511dc10a1e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.081473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3897ef-be9e-421a-9860-2ec833ed5c1f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.111003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c44fe-5871-4a56-90da-6809f95e4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.111565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c44fe-5871-4a56-90da-6809f95e4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.111902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c44fe-5871-4a56-90da-6809f95e4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.112174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c44fe-5871-4a56-90da-6809f95e4ec2 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.113246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c44fe-5871-4a56-90da-6809f95e4ec2 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.114661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c44fe-5871-4a56-90da-6809f95e4ec2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:03.117238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.117869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.118146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.118416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] PATH_INFO: `/v3/projects/127332367fa24e318828c23b1f6090cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.120376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] RBAC: Authorizing `identity:get_project(project_id=127332367fa24e318828c23b1f6090cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.121726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 219/438] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:03.121833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:03.122699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.122977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.123246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.127796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 220/439] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:16:02 2022] GET /identity/v3/projects/127332367fa24e318828c23b1f6090cc => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:03.127796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.143879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:03.145042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:03.202022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.221205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 220/440] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:03.222345 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.238003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f919eea-8359-4844-9ac0-09ab708263f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.272856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.273459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.273749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.274013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.275465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=660e237685d342e8bb6bef00916bca0b, scope.project.id=13c11b499ec641ce8a2a829378a48649) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:03.275819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.277388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:03.287696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 221/441] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=660e237685d342e8bb6bef00916bca0b => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:03.287696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.346495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.353566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf30921-7987-41a3-b346-aca3418c452a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.396559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.397219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.398010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.398427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.400540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.402244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:03.408696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 221/442] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:03.409822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.425871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649add02-4ae4-46d5-8034-8f42857da2c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.463105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.463961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.464238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.464508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.473777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.479144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:03.485261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:03.486164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:03.486974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 222/443] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:03.492533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.535717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95400bd-fdea-411f-a47b-7d7ca21a84fb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.574507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.575115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.575399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.575702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.576431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=127332367fa24e318828c23b1f6090cc) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:03.576737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.578179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:03.585350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 222/444] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=127332367fa24e318828c23b1f6090cc&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:03.586590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.608835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbda879-5f3d-4518-8272-34dee5065d81 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.625355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.652109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.652744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.653028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.653305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.654010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:03.654318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.655787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:03.679485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 223/445] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:03.680524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.732162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007e4da-63b3-41a5-b860-b6b40d233e31 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.764901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.786152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.786152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.786152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.786152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.798836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:03.804800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:03.805721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:03.807135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 223/446] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:03.812814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:03.826415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619780b5-6bc9-42b7-b048-606a7ab84775 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.877315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:03.877935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:03.878213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:03.879088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:03.942411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:03.963711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66099a7d-6232-46f1-bf9e-f7303e4d1021 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:03.979158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:03.981121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.013659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.014309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.014591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.014865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.015526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:04.017037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.018431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.030117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 224/447] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:04.036072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.074927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:04.130052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 224/448] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:04.135898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.168685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2c7c3-74f4-42ab-ac8d-160fdad7224e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.202590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.203126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.203402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.203704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.222093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:04.239121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d69596-65fd-409b-8a60-194b593fa82e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.271262 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.272763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.275249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.276013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.276293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.276569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] PATH_INFO: `/v3/projects/df82cd1e5393473c92978bf22c792e3e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.277689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] RBAC: Authorizing `identity:get_project(project_id=df82cd1e5393473c92978bf22c792e3e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.279114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.280758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 225/449] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/projects/df82cd1e5393473c92978bf22c792e3e => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:04.286118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.313291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:04.374859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 225/450] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:04.380457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.383733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb84ed-42d6-40b5-89b1-c9b2500ddeb4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.417927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.418553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.418832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.419107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] PATH_INFO: `/v3/groups/ea422491970a4b37a9cd16b546561f73` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.420249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] RBAC: Authorizing `identity:get_group(group_id=ea422491970a4b37a9cd16b546561f73)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.421693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.423248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 226/451] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/groups/ea422491970a4b37a9cd16b546561f73 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:04.428331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:04.429020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.478261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c1dc0-caf3-4905-b6a2-c851099959d5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.511000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.511654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.511937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.512208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] PATH_INFO: `/v3/projects/13c11b499ec641ce8a2a829378a48649` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.513344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] RBAC: Authorizing `identity:get_project(project_id=13c11b499ec641ce8a2a829378a48649)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.514777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.516577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 226/452] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/projects/13c11b499ec641ce8a2a829378a48649 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:04.521927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.537913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:04.544265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef7a5-1c6f-4a58-9f34-2864fcc729d2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.579035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.580161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.580443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.580721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.581529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] RBAC: Adding query filter params (group.id=ea422491970a4b37a9cd16b546561f73, role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=df82cd1e5393473c92978bf22c792e3e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:04.581840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.583263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.589995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 227/453] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/role_assignments?group.id=ea422491970a4b37a9cd16b546561f73&scope.project.id=df82cd1e5393473c92978bf22c792e3e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:04.626877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fa32e-593c-4a57-98e3-483832ca800c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.645928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:04.663788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1cd4a-321c-4c07-bf03-e9050b082219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.664453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1cd4a-321c-4c07-bf03-e9050b082219 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.664705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1cd4a-321c-4c07-bf03-e9050b082219 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.664968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1cd4a-321c-4c07-bf03-e9050b082219 None admin] PATH_INFO: `/v3/groups/c25a751122844af9bff9e5a68900901a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.666221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1cd4a-321c-4c07-bf03-e9050b082219 None admin] RBAC: Authorizing `identity:get_group(group_id=c25a751122844af9bff9e5a68900901a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.667692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1cd4a-321c-4c07-bf03-e9050b082219 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.669290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 227/454] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/groups/c25a751122844af9bff9e5a68900901a => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:04.675432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:04.782219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e37e-1b77-4c4c-bd65-c5069598b263 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:04.817197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:04.817795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:04.818074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:04.818441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:04.819176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] RBAC: Adding query filter params (group.id=c25a751122844af9bff9e5a68900901a, role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=13c11b499ec641ce8a2a829378a48649) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:04.819486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:04.820971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f112d-a7c5-44c1-b489-984dd900e8e0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:04.827262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 228/455] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Sep 9 06:16:04 2022] GET /identity/v3/role_assignments?group.id=c25a751122844af9bff9e5a68900901a&scope.project.id=13c11b499ec641ce8a2a829378a48649&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:07.533507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf2ebe-2864-41e6-aba6-305065787c76 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:07.534022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf2ebe-2864-41e6-aba6-305065787c76 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:07.534022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf2ebe-2864-41e6-aba6-305065787c76 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:07.535761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 228/456] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:07.541210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1c9a4-eff2-4a87-8ced-a9388a68bad3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:07.542134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1c9a4-eff2-4a87-8ced-a9388a68bad3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:07.542604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1c9a4-eff2-4a87-8ced-a9388a68bad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:07.569427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cfa1c9a4-eff2-4a87-8ced-a9388a68bad3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:07.570959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cfa1c9a4-eff2-4a87-8ced-a9388a68bad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:07.600985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:07.671897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 229/457] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:07.750788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84655aa8-fb8b-41f7-a3f6-6dedd314182b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:07.751091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84655aa8-fb8b-41f7-a3f6-6dedd314182b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:07.751342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84655aa8-fb8b-41f7-a3f6-6dedd314182b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:07.753067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 229/458] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:07.769192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:07.769477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:07.769750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:07.798530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:07.799692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:07.901871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 230/459] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:07.986883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84655aa8-fb8b-41f7-a3f6-6dedd314182b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.128150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84655aa8-fb8b-41f7-a3f6-6dedd314182b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.173740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.174732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.175252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.175750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.183792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.191309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:08.192808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8d8e4217-7052-4f14-9da3-192d430df5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:08.208934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:08.209979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:08.210849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:08.210849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:08.210849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:08.210849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 230/460] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:08.211581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ee38-45df-42ef-92f5-5c74b10e7ffc None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.211802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ee38-45df-42ef-92f5-5c74b10e7ffc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.212257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98ee38-45df-42ef-92f5-5c74b10e7ffc None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.213876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 231/461] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:08.220428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.220715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.220995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.241153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:08.242400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:08.276355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:08.289649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:08.306168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd9cc-1888-42dd-95d8-ac0412423b87 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.319639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 232/462] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:08.320778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.345790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.346429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.346706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.346973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.365061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:08.367247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-34c97861-8249-4bab-96d9-48c6a4f26756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:08.384729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:08.385843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:08.386837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:08.386837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:08.386837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:08.388658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 231/463] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:08.394579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6a76-bc22-4f6a-81c6-11a6dbd6fb86 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.395077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6a76-bc22-4f6a-81c6-11a6dbd6fb86 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.395517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d6a76-bc22-4f6a-81c6-11a6dbd6fb86 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.396953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 232/464] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:08.397781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6a76-bc22-4f6a-81c6-11a6dbd6fb86 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.451899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8030c-e997-4a68-9f13-86149901cc69 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.487909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.488608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.489054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.489482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.490264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:08.490726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:08.492359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc653cad-4205-42c6-a122-7eb73b9de0c7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:08.504898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d6a76-bc22-4f6a-81c6-11a6dbd6fb86 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.505330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 233/465] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:08.506924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.507433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.507879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.527364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:08.528880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:08.540641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.541027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.541354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.541894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.542297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:08.542753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:08.544374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:08.547092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:08.558290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 233/466] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:08.559265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.562917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:08.621714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 234/467] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:08.622958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.682186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ddaba-4164-4545-aabc-ba2384d43df9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.728888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.730037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.730456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.731301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.732396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:08.734598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:08.740088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47517d05-52ac-4798-9d08-89cd6bd56005 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.768349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 234/468] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:16:08 2022] POST /identity/v3/users => generated 246 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:08.770275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.786207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.787205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.787829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.788308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.789177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:08.791785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:08.827959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 235/469] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:16:08 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:08.856895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:08.944183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5918f-9dbd-44b7-939c-947d644f4701 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:08.959684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:08.991756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:08.992537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:08.992832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:08.993104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:08.995181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:09.010868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:09.012510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:09.044682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:09.045677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:09.046900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:09.046900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:09.046900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:09.046900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 235/470] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:09.054748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:09.141718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5466959-30a0-47cc-8dd8-5e198521e3b6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:09.183047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:09.183748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.184354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.184633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.187096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1beb-eb91-4fb7-a2cf-3077788f49bb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:09.189742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:09.197245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:09.198742 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:09.214908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:09.215907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:09.216765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:09.216765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:09.216765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:09.216765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 236/471] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:09.223497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:09.234503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:09.235176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.236139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.236420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.237082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:09.237387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:09.239354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:09.257127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 236/472] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:09.263097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:09.309082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:09.388035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb8f6f-4e60-48c8-a99a-c7a56f3f7148 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:09.422358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bf1fb-eb73-432d-9f5a-eb754076482b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:09.442756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:09.443775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.444445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.444789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.445519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:09.445890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:09.447732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac4e23-f12d-40f7-a9a9-5a86e9897983 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:09.465661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e78a0c-1f3b-4fc5-b2c0-5be797936c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:09.466724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e78a0c-1f3b-4fc5-b2c0-5be797936c35 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.467062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 237/473] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:09.467475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e78a0c-1f3b-4fc5-b2c0-5be797936c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.468027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e78a0c-1f3b-4fc5-b2c0-5be797936c35 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.468495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928e152-9c8c-4fca-9599-2a5092a438bd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.468780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928e152-9c8c-4fca-9599-2a5092a438bd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.469058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928e152-9c8c-4fca-9599-2a5092a438bd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.469504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e78a0c-1f3b-4fc5-b2c0-5be797936c35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:09.470581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 238/474] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:09.471277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928e152-9c8c-4fca-9599-2a5092a438bd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:09.471658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e78a0c-1f3b-4fc5-b2c0-5be797936c35 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:09.501752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 237/475] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:16:09 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:16:09.503423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.504612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.505166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.531505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:09.533418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:09.552998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:09.644074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928e152-9c8c-4fca-9599-2a5092a438bd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:09.652106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 238/476] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:09.679356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:09.690783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:09.691860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:09.692399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:09.692893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:09.693769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:09.695443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:09.724299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 239/477] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:16:09 2022] POST /identity/v3/users => generated 249 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:09.971106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:10.088552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845183f-d441-4873-81fc-90fb75b34163 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:10.127876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:10.128494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:10.128775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:10.129048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:10.129727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:10.131097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:10.144571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 06:16:10.144571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 06:16:10.144571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [parameters: {'id': '59f5886852444e33aadbb9dd4ef13db2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 06:16:10.144571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Sep 09 06:16:10.161834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 06:16:10.162764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 06:16:10.164130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 06:16:10.165201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 06:16:10.166137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 06:16:10.166974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'id': '59f5886852444e33aadbb9dd4ef13db2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:10.169443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 06:16:10.171409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:10.177970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 239/478] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Sep 9 06:16:09 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:10.179573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:10.332051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaad7d-7f4b-4009-9990-5e65718909c1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:10.378679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4029-9699-4ae3-a8be-3b2ec46dd9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:10.379243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4029-9699-4ae3-a8be-3b2ec46dd9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:10.379601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4029-9699-4ae3-a8be-3b2ec46dd9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:10.379882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4029-9699-4ae3-a8be-3b2ec46dd9a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:10.393202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e4029-9699-4ae3-a8be-3b2ec46dd9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-198e4029-9699-4ae3-a8be-3b2ec46dd9a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:10.398878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:10.399899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 06:16:10.400758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:10.407651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 240/479] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:10.412631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:10.444720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:10.577269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763e336-0a94-4999-8917-491c459cf44c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:10.620411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:10.621021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:10.621298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:10.621569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:10.622237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:10.622539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:10.623947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245eb36-2a5a-4029-a95d-b51c5b628f1d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:10.632731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 240/480] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:11.637264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792228-f07a-4029-9283-22a4db9a3ec2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:11.637774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792228-f07a-4029-9283-22a4db9a3ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:11.637849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792228-f07a-4029-9283-22a4db9a3ec2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:11.639404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 241/481] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:11.645020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b35a9-ac30-4c47-90b1-8b672353e40a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:11.645863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b35a9-ac30-4c47-90b1-8b672353e40a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:11.646337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b35a9-ac30-4c47-90b1-8b672353e40a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:11.648362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 241/482] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:11.648550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:11.650693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:11.650693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:11.662173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:11.662767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:11.663237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:11.672829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:11.674146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:11.695812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:11.697302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:11.724918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:11.785959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 242/483] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:11.821222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 242/484] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:12.208068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.234839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.243996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:12.355922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23149517-ac0c-481f-8c4e-847e82e6275d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:12.363141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100d5b9-293e-4a0d-ae96-6cec32769258 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:12.393859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa76b9-cdb7-48f5-9bab-f43053dc3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:12.394730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa76b9-cdb7-48f5-9bab-f43053dc3a94 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.395152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa76b9-cdb7-48f5-9bab-f43053dc3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.395584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa76b9-cdb7-48f5-9bab-f43053dc3a94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.405775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf8ff5-5bc3-40ef-9f50-580bafe236cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:12.406412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf8ff5-5bc3-40ef-9f50-580bafe236cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.406695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf8ff5-5bc3-40ef-9f50-580bafe236cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.406972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf8ff5-5bc3-40ef-9f50-580bafe236cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.408467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa76b9-cdb7-48f5-9bab-f43053dc3a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3faa76b9-cdb7-48f5-9bab-f43053dc3a94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:12.413647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:12.414746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:12.415994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:12.422312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf8ff5-5bc3-40ef-9f50-580bafe236cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:12.422574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 243/485] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6fcf8ff5-5bc3-40ef-9f50-580bafe236cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:12.423328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:12.424418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:12.425400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 243/486] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:12.426214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd040ae0-5829-4115-8871-7b0c773e796a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.426214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd040ae0-5829-4115-8871-7b0c773e796a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.426405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd040ae0-5829-4115-8871-7b0c773e796a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.427194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888321d1-340d-4f6b-b5dc-5480393ca9f7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.427479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888321d1-340d-4f6b-b5dc-5480393ca9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.427680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 244/487] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:12.427774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888321d1-340d-4f6b-b5dc-5480393ca9f7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.431619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd040ae0-5829-4115-8871-7b0c773e796a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.431766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 244/488] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:12.434900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888321d1-340d-4f6b-b5dc-5480393ca9f7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.469591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:12.507316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:12.545768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888321d1-340d-4f6b-b5dc-5480393ca9f7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:12.547665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd040ae0-5829-4115-8871-7b0c773e796a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:12.584497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:12.585248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.585532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.585798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.586450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:12.587355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:12.588935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3a663-6258-412d-96d6-124c3eea50d8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:12.589401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:12.589997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.590468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.590724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.591827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:12.592134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:12.593497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0eacc-52d0-4f2a-b9d8-cd42d8fbfd27 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:12.599641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 245/489] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:12.601816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.602394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.603379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.612751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 245/490] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:12.614414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.614693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.614961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.624594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:12.625848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:12.636773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:12.637967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:12.699607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 246/491] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:12.699607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.731012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 246/492] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:12.731922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.748179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:12.838687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:12.867140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0484030-8ff7-4628-a523-28b6c2d643dc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:12.908861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:12.910390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.911005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.911663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:12.923187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92391bd9-a018-4b80-b597-8b755bb16067 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:12.934077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:12.935840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:12.958811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:12.959802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:12.960666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:12.960666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:12.960666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:12.968659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 247/493] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:12.975537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:12.992711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:12.995456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:12.995981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:12.996452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.015875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.017398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:13.044687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:13.045649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:13.046517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:13.046517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:13.046517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:13.046517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 247/494] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:13.046730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.075021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.105240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a0c6-fda5-431e-a034-6918b3783fc8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.150395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.151051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.151627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.151627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.152265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:13.152614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.155144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.163798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59452c06-cefd-4c5b-b683-aa62e6e06dcd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.172653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 248/495] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:13.172653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.184324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.220241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.220875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.221218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.221500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.233961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:13.239264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:13.240360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 248/496] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:13.241152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.303303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a9c07-f15d-4578-9f78-ba3c692f6f22 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.340706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.341313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.341589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.341856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.342491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:13.342778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.344196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.347016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.348462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d042a-c63d-4be6-a1bf-9c1138ffd4ef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.357614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 249/497] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:13.358450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.389365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.390276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.390744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.391092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.401975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:13.407186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:13.408118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:13.408969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 249/498] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:13.409656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.446880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.480522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6004d4d-934e-4f3b-a7ea-8337e75953a9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.516978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.518055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.518341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.518618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.527318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.528697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.531244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875c2ab-c833-4a6c-a6c0-05fc42d569eb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:13.541772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:13.542719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:13.543743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 250/499] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:13.544300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.572233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.572773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.573049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.573578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.573983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:13.574284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.575665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.583957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 250/500] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:13.584589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.601597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.645756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678150e1-897b-4f8d-8d0d-879dfbc60e8c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.678712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.679451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.680229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.680720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.681180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.687819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0564-410c-49d3-8f4a-022a70e2bca8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.688647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.690102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:13.701752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:13.703159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 251/501] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:13.704085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.726067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.726670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.726946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.727241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.728270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:13.728579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.729951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e61c70-e96e-40b2-bf1b-a6c8fbd61ccd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.737789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 251/502] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:13.738935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9da5-277a-488d-affb-65043a75d49b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.739217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9da5-277a-488d-affb-65043a75d49b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.739487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9da5-277a-488d-affb-65043a75d49b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.740697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 252/503] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:13.741385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9da5-277a-488d-affb-65043a75d49b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.813491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a747bb-ec5a-4690-8ec6-9528ef6c822d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.831410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.841208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9da5-277a-488d-affb-65043a75d49b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:13.848669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.849277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.849560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.849841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.850563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:13.850871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.852368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.873956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:13.874572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:13.874847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:13.875116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:13.886887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:13.888388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:13.889538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 252/504] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:13.890384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:13.903141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:13.904155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:13.905056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:13.905056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:13.905056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:13.905056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 253/505] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:13.905056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:13.917123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:13.996176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1a0bb-6e90-4a83-901d-e54bc16e2e7d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.007121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cc53f-55dd-4751-a359-a9bf8708f5b8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.031969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.032578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.032858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.033129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.033855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:14.034161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.035634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639a4d7-e8da-4eaa-97a4-ec8b03231101 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.039996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.040609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.040884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.041151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.051314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.052305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 253/506] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:14.052773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.053493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.053776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.054049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:14.069201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:14.070164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:14.072099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:14.072099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:14.072099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:14.072099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 254/507] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:14.072099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.076436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:14.077706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:14.104024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.132656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.146000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 254/508] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:14.146920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.177738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a008bda-8457-48ab-a7de-af62ca9b6543 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.213888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.214500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.214784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.215060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.261131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917a26d-5b69-4846-adc1-7827a4c0eb6f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.286662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.288619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.299002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.299644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.299925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.300200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.300841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:14.301146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.302505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.315582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 255/509] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:14.315720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.342900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.359962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.408122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 255/510] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:14.408979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.425899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8d9f-e0a7-4825-bed5-48c96bbf7a11 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.458792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.459388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.459694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.459964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.506102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0581390-1d1a-4ee9-9cb5-1091edcb026c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.528949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.530408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.540816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.541421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.541694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.541961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.542605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:14.542912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.544291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.554724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.556319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 256/511] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:14.557329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.576680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.638704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 256/512] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:14.639482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.663848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568f7f8-8239-43e7-aed9-755b25fe8cc7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.697706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.698338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.698621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.698901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.706521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.708026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:14.720631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:14.721578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 257/513] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:14.722992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.737931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c17dd-a54e-45bd-890a-5757dc7c2d63 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.760344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.771862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.772490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.772765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.773034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.783450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.784920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:14.799450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:14.800455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:14.801318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:14.801318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:14.801318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:14.801482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.801877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 257/514] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:14.801877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.828968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4245b3-3533-4c37-882e-16e2f755ec74 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.863722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.864351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.864634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.864925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.875990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.877433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:14.892811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:14.893742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:14.894628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:14.894628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:14.894628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:14.894628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 258/515] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:14.894628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.907163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d1ea7-abf4-4c9f-95ba-09b6781a6bc4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:14.941312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:14.941938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:14.942216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:14.942491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:14.950254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:14.951747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:14.963261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:14.964373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 258/516] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:14.965488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:14.999095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:14.999396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588b0ed-2c45-4af8-afca-b7506a67457d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.028217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.034084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.034746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.035028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.035303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.046396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.047908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:15.062102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:15.063113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:15.064029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:15.064029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:15.064029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:15.065378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb2f93-4965-4a16-a3e8-2b64e167c280 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.067826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 259/517] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:15.068534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.099707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.100393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.100740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.101021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.101746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:15.102053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.103671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.120587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 259/518] 10.222.0.25 () {56 vars in 1139 bytes} [Fri Sep 9 06:16:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:15.121306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.170036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d7c4-0399-425c-8f65-cc95d001c154 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.203749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.204614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.204897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.205173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.205823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:15.206126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.207504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.212503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.220025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 260/519] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:15.220651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.221280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33893a77-cf99-41d8-b640-4f455888a9d7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.256118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.256720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.257001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.257276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.257913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:15.258273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.260205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.267274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.271494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 260/520] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:15.272134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.329110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3edfa-8094-4187-adad-df418d56ea8b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.363663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.364278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.364557 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.364826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.365557 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:15.365863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.367325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.370889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d26dc-6326-4341-86df-ac193f3e28ad None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.403896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 261/521] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:15.404341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.404726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.405035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.405318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.405591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.406230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:15.406530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.407918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.420138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 261/522] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:15.420781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.435322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.506181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.509429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e5db-fb0e-4e94-a660-4a6cfbce3a31 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.520849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c002aac-c1d7-436d-9771-fbdd2a438dbe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.544982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.544982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.544982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.545331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.554695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.555986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.556275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.556547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.557181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.558622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.579929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 262/523] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:16:15 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:15.580761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.620619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.622527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.663227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.706226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e7fe-4121-4b15-82cc-2fa79117ca40 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.732070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.740424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 262/524] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:15.740424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.752136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.752781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.753061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.754070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.762768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.764491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:15.777157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:15.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:15.778927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:15.779348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 263/525] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:15.780004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:15.854456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809066f-9f59-4301-8d62-ba102323b29e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.889243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.889857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.890141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.890412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.895519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4bbd-71e4-4839-aec4-5d193a2f9732 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:15.925352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.936090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:15.936729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:15.937118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:15.937399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:15.945190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.946846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.957845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:15.960359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:15.961676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:15.962490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:15.968271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:15.968712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:15.968784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 264/526] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:15.968784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.078364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df7306-840d-4039-b42a-d8699a062223 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.083225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 263/527] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:16.083225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.118998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.119662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.119947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.120218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.131231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.132756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:16.149432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:16.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:16.151270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:16.151270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:16.151270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:16.151270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 265/528] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:16.151489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.194017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:16.197066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:16.197189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf169b1-a704-4042-acfd-50f69a4792bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.235930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.236465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.237063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.237345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.238202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:16.238514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.240209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.263886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 264/529] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:16.263886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.264186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00853db-73c6-4030-9769-c3fff1dfe41a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.299181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.299826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.300324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.300606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.302079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:16.302391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.303835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.326854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 266/530] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:16.328055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.373558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb1395-7a71-4019-b521-c4dbafa1047b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.412998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.413632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.413916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.414190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.425037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.426582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.434270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:16.438252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8624a-05c0-41d7-93fe-08c1488e76a0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:16.441637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:16.442549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:16.444020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:16.444020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:16.444020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:16.444020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 265/531] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:16.444020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.460563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:16.475421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.476043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.476325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.476595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.477772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:16.478325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.479724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.491455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 267/532] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:16.492258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.565006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08cb52-2372-4517-8b3c-c4512806c9da None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.602969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.603590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.603897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.604143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.604993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=0f8bf35f310b4b558dde597cde49655f) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:16.605419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.606843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.608164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93750c4-4e2b-4fdb-9cf8-876017c16c4c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.623631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 266/533] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/role_assignments?user.id=0f8bf35f310b4b558dde597cde49655f&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:16.623631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.661161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.661798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.662079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.662353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.663105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=dc56896d10094aed91db84ea2bf9dfb2) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:16.663416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.665247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.688053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 268/534] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/role_assignments?user.id=dc56896d10094aed91db84ea2bf9dfb2&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:16.689078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.736372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:16.757054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:16.794636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fbca-0f0b-4be3-ac78-69aa7b084bd9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.870088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e62fbc-0321-4704-8086-0ffb5c7a4caa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:16.893066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.893733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.893974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.894245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.894904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:16.895206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.896632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.917310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:16.917960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:16.918233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:16.918501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:16.923228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 267/535] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:16.929640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:16.931766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:16.936574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:16.943972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:16.944839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:16.946326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 269/536] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:16.953683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:17.123573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a8d19-004f-411a-b26e-cebdf1f850b8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:17.126720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:17.155792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a058-0655-4a53-9a2d-228ad79ea278 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:17.195153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:17.199808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.200111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.200387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:17.205597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:17.213255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:17.214845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:17.224357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:17.224998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.225339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.225625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:17.226336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:17.226646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:17.228182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:17.229222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:17.230453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:17.231269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:17.231339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 268/537] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:17.235032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:17.254088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 270/538] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:17.261223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:17.423148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e390ea-b1e2-4d59-8b24-861fc725fc58 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:17.427538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebd55b-05d2-47ff-b74d-bc5195d0f28f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:17.471001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:17.471641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.471924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.472196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:17.472900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:17.473204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:17.474644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:17.491717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 269/539] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:17.504717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:17.505234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:17.505982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.506263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.506537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:17.507348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=86cd9a25b7ed4206a9b5939024168475) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:17.508044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:17.510045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7af807-110b-4723-a1a9-ea78d666eb0d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:17.517326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 271/540] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:17 2022] GET /identity/v3/role_assignments?user.id=86cd9a25b7ed4206a9b5939024168475&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:17.575140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:17.622991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822ff5-dab3-4d32-b516-69fc14fe2692 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:17.660373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:17.661012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.661292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.661574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:17.662328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=f236664fa09b4a4aafe6980768fc263b) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:17.662640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:17.664153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a31e7-e60b-419b-930f-f07c57892c54 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:17.670866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 270/541] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:17 2022] GET /identity/v3/role_assignments?user.id=f236664fa09b4a4aafe6980768fc263b&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:17.973497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aa71-87a0-4391-b3ba-966faaefe17d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.974348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aa71-87a0-4391-b3ba-966faaefe17d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.974775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aa71-87a0-4391-b3ba-966faaefe17d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:17.976558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 272/542] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:17.983253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:17.983758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:17.983852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:18.009066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:18.010415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:18.099702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 271/543] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:18.438247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7aa71-87a0-4391-b3ba-966faaefe17d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:18.450097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:18.559605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7aa71-87a0-4391-b3ba-966faaefe17d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:18.608789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:18.609428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:18.609708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:18.609986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:18.629411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:18.638542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:18.639533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:18.640402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 273/544] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:18.648461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:18.723488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:18.787081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd657f1-b939-4853-a11b-a1f49e7616b7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:18.832686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:18.833303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:18.833581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:18.833848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:18.834520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:18.834822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:18.836205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:18.845051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 272/545] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:18.853964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:18.941465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:18.986443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932a93-c732-4f49-9093-e3025923f9e0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:19.034567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:19.035211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.035494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.035794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.048707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:19.050172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-91a8a1d8-5fe4-4065-9d29-258cb090de94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:19.069831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:19.070929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:19.071941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:19.071941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:19.071941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:19.075283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 274/546] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:19.076586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:19.115681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e922db-fe41-461b-abd5-2c266800c1ef None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.115997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e922db-fe41-461b-abd5-2c266800c1ef None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.116217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e922db-fe41-461b-abd5-2c266800c1ef None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.118745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 275/547] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:19.123847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.124133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.124411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.142610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:19.146120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:19.220395 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bae30-0785-48ab-95fe-d06651905893 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:19.248855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:19.284332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:19.284960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.285244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.285516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.286172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:19.286472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:19.287882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f7141-3526-48a5-86ea-9631e7d02a61 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:19.291846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 276/548] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:19.303244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 273/549] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:19.309457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:19.377027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:19.377210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a376f7-b2a6-4c01-b053-1c70c4358e12 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.377437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a376f7-b2a6-4c01-b053-1c70c4358e12 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.377638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a376f7-b2a6-4c01-b053-1c70c4358e12 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.378973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 274/550] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:19.397017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.397298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.397575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.436300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5327e-8795-47d8-90c6-a1b9267cea32 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:19.436914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:19.437774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:19.484599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:19.485251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.485548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.485806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.506223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:19.507629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:19.527382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:19.528566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:19.529506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 277/551] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:19.534520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:19.575568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 275/552] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:19.579774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:19.696267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387ef91-3bd6-41e3-aae9-202d9a44e3ce None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:19.707875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:19.735196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:19.736196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.736684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.737209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.738155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:19.738664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:19.740482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:19.761271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 278/553] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Sep 9 06:16:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:19.766957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:19.844627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a505f3-6989-4319-9a01-27ff44063960 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:19.847834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:19.881570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9160bf-15d9-4b7d-8bee-7d140e2dbb3f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:19.889579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:19.890220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.890501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.890771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.901836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:19.907806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:19.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:19.909655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 276/554] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:19.910357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:19.933751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:19.934609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:19.935068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:19.935489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:19.984285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.026720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589850e-9d5b-45f9-b32c-337aaefd2d0f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.033850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.036413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.064170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.064990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.065371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.065817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.078274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:20.083815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:20.084954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:20.085994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:20.088522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 277/555] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:20.089479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.116721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.183781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 279/556] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 06:16:20.184876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.210086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3087-1d34-4d24-8caa-57e2d15df301 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.250672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.251735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.252252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.252730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.253636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:20.254154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.255859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.260398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.266887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 278/557] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:20.268212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.298555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18eab-2f4e-4947-8c67-3dc03da9bfac None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.335765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.336844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.337138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.337601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.338064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:20.338370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.339766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.349651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 280/558] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:20.350546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.365025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.386664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ce6b1-60ff-4000-8767-40a8a3380aeb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.422943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.423619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.423907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.424180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.436858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.438301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:20.452953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:20.454129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:20.455355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:20.455355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:20.455355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:20.455355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 279/559] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:20.455631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.467905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8e178-b12b-40c0-b844-e2105cd4cdf1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.498096 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.503961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.504599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.504899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.505153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.516061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.517504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-021090c6-0e0f-4ac2-a920-dd186adcc6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:20.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:20.533278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:20.534186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:20.534186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:20.534186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:20.534186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 281/560] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:20.534756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd877-858d-44de-8036-dfd10ac95101 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.535034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd877-858d-44de-8036-dfd10ac95101 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.535330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fd877-858d-44de-8036-dfd10ac95101 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.536714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 282/561] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:20.537275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fd877-858d-44de-8036-dfd10ac95101 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.562591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fcd35-3e3f-4d95-a26b-0a613b1f416c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.600031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.600666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.600970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.601241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.604446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.613053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.614485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f54caef5-a966-4514-b211-be2e1a3803ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:20.628974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:20.630065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:20.631011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:20.631011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:20.631011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:20.631011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 280/562] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:20.631262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d059f8-e251-4dc5-b2cb-ddd993f97dc0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.631478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d059f8-e251-4dc5-b2cb-ddd993f97dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.631764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d059f8-e251-4dc5-b2cb-ddd993f97dc0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.632922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 281/563] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:20.634159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d059f8-e251-4dc5-b2cb-ddd993f97dc0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.646220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fd877-858d-44de-8036-dfd10ac95101 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.686050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.686656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.686935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.687211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.687884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:20.688189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.689548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c4976-01d2-4d0a-9c8a-c28acf98dae7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.701821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 283/564] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:20.703008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.703293 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.703595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.720984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:20.722177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:20.741374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d059f8-e251-4dc5-b2cb-ddd993f97dc0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.757437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.775272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.775903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.776190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.776461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.777100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:20.777402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.778761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533e88b-76d6-41f7-8754-a5215cc72ca7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.788973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 284/565] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:20.789806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.794746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 282/566] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:20.794746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.794746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.794746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.817163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:20.818550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:20.865957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:20.895631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 283/567] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:20.895631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:20.918023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aad3f-5215-4482-ac65-4e817c783e16 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:20.957632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:20.959214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:20.959723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:20.960194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:20.961841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:20.962452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:20.964234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:20.979994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 285/568] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:20.981017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.009131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580f66f-38b4-45ae-9de0-6e884748eeb4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.019300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.050705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.052282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.052788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.053497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.063852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.065320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:21.077705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:21.079057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 284/569] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:21.080597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.105152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a89c-ecb4-4038-80ba-2f12ee93b3a6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.119410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.142895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.143526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.143831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.144107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.152054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.153503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:21.165001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:21.166145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:21.167134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 286/570] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:16:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:21.167778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.188818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ce61d-3eb1-44f6-a7d1-0af8caa141c8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.224335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.224963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.225244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.225518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.232561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.234367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:21.245989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:21.247124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 285/571] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:21.248437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.262869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.272732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3130ea-474b-4eca-a6e9-0da15c16c461 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.307883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.308509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.308786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.309047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.319562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:21.325458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:21.326573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:21.327624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:21.331573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 287/572] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:21.332334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.344660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.348244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724a5f-cf14-49df-a467-1c58786ee5a4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.382220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.382826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.383679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.383965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.384657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:21.384963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.386378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.397940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 286/573] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:21.398729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.431329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a8fb6-76c2-402f-b1da-59108bab2cef None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.464852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.465453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.465722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.465987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.466702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:21.467001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.468444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.485179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 288/574] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:21.486097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.498371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f967-c3f7-4efb-bd71-d09f6939f9de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.498518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.533512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.534136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.534412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.534680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.545674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:21.550519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:21.551649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 287/575] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:21.552794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.574080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.593013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d0dd7-4e31-4ba0-939a-732ed5fa7a7e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.627284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.627928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.628197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.628461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.629173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:21.629475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.630889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.647406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 289/576] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:21.648260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.661711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e3fc2-1817-42fb-93b0-da1b2cb43957 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.697044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.697646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.697924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.698193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.698851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:21.699151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.700539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.708436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 288/577] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:21.709204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.715284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.748916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b931d2e-e253-4496-947d-063fc0e802e7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.783078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.783710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.783991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.784265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.784984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=cb474b7f08db4ff9b9fdf8eb8ca658af) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:21.785290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.786711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.791580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.791972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 290/578] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/role_assignments?user.id=cb474b7f08db4ff9b9fdf8eb8ca658af&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:21.792757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.831792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4f474-9eb5-40bb-a621-5b42b525d962 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.871686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.872612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.873039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.873473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.886377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.888454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:21.906745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:21.907986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:21.914216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:21.914216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:21.914216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:21.914216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 289/579] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:21.914216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:21.918600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0d7c-a19d-4fe8-9641-c164328479c4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:21.953325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:21.960247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:21.961025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:21.961914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:21.962391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:21.963345 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:21.963733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:21.965230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:21.975566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 291/580] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:21.976508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.022923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179a1e-8150-424d-8ab3-67c798edd357 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.052772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.059122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.059792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.060078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.060350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.071069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.072754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:22.087813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:22.093900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:22.095024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:22.095024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:22.095024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:22.095024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 290/581] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:22.095024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.098668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb88c1-f3a0-4b96-b7fe-19d244277ec7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.136779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.137387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.137595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.137865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.149360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.150968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:22.166170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:22.167162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:22.168130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:22.168130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:22.168130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:22.168130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 292/582] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:22.168880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.203021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf18a6-a191-4c62-9b0b-fe60c76f06f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.218842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.238795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.239415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.239718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.239986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.240635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:22.240938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.242306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.254823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 291/583] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:22.255479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.283332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f02395-042c-4bdc-b5d5-018f51db8d87 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.305451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.322048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.322799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.323147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.323496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.335202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.336844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:22.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:22.353546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:22.354642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:22.354642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:22.354642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:22.356196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 293/584] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:22.357145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.370031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386972-eaeb-4afa-bc0f-f2d900e161eb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.409089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.409696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.409979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.410257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.410904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:22.411238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.412659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.425146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 292/585] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:22.425969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.463423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.468228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2bfc5-4128-488c-9a3c-a61741a467ac None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.505378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.506003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.506296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.506567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.507210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:22.507520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.509618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.521334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 294/586] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:22.522165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.550010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07782a8b-ce1d-4c99-849e-48dc74a46861 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.553585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.591894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.592531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.592813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.593089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.600026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.601433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:22.614248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:22.615180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 293/587] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:22.616167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.636925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8e099-ebf3-4f0d-9f2e-8a8de019f146 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.674676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.675225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.675506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.675815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.676476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:22.676866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.678276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.695370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 295/588] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:22.695370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.714095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.736916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89736b7d-2f76-43b2-b9fb-893177876e0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.770722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.771349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.771658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.771932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.779328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.780802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:22.792286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:22.793346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:22.794555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:22.795070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 294/589] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:22.795953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.807967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1aa80-3cff-45f2-96ca-13f7e49118ff None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.822850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.843346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.844012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.844291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.844566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.852560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.854006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:22.865467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:22.866600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 296/590] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:22.872813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.911381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf540d2-e5ae-469d-b6a7-ba9808a6a3fd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:22.946276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:22.946810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:22.947089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:22.947358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:22.948092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:22.948399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:22.949851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:22.957571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:22.968301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 295/591] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:22.969018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:22.982101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eec86-1c73-436a-a7a0-ad887e7e51a7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.024854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.025493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.025774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.026046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.033617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.035056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:23.046449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:23.047647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:23.048604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 297/592] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:23.049258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.063949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:23.080482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580d379-ec0a-4bcb-82ae-ee09442c2824 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.113953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.114595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.114890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.115169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.115893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:23.116199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.118237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.130401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 296/593] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:23.132507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.168694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfba758-e7df-4786-9cdc-db319b8c2ee2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.205258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.205796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.206080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.206356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.207143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:23.207456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.208904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.219681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:23.230371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 298/594] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:23.235796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.249320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b1d15-b08e-4de8-b024-31638d5d0c76 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.284520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.285182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.285494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.285742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/dc56896d10094aed91db84ea2bf9dfb2/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.300750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=dc56896d10094aed91db84ea2bf9dfb2, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.302173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.304448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:23.317452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 297/595] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:23 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/dc56896d10094aed91db84ea2bf9dfb2/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:23.328923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.349679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a01d43-50f8-42f9-821d-bd36156128f5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.394092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.394961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.395346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.395804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.396722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:23.397131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.399475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.421999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 299/596] 10.222.0.25 () {56 vars in 1139 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:23.423013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.453807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdab9-6e4c-4e60-b4f1-fbfdb3de5d5d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.488060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:23.494515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.495455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.495914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.496328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/0f8bf35f310b4b558dde597cde49655f/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.514650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=0f8bf35f310b4b558dde597cde49655f, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.516359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.539662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 298/597] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:23 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/0f8bf35f310b4b558dde597cde49655f/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:23.540870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.553523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b2489-a8fe-4a1a-b1b3-3bf2fc89780f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.628167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.628994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.629372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.629752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.644374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.646313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.647919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:23.674581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:23.675840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:23.677030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:23.677030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:23.677030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:23.678620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 300/598] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:23.684643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.782916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2d8e9-fb92-4a4e-871d-85f85d5ea4bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.795427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e360243-74fb-480b-88eb-6d811dfea83a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:23.802719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:23.830940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.831571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.831856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.832128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.832769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:23.833072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.834446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.837429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:23.838088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:23.838376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:23.838651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:23.853191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 301/599] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:23.854731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:23.863870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:23.865472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:23.897789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:23.898968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:23.899919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:23.899919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:23.899919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:23.905449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 299/600] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:23.908666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.026705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a9f6f-b278-4c3b-bf80-042494305be5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.061007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:24.067711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacf11d-530f-4d74-9127-c4c15e51ad29 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.077613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.078906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.079604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.080497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.089518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:24.090983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a0d0b358-53eb-4d64-b726-2c7070201153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:24.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:24.103981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:24.105012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 302/601] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:24.105512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905c04c-e108-4ac1-aa61-1d5d83e99088 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.105834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905c04c-e108-4ac1-aa61-1d5d83e99088 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.106040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905c04c-e108-4ac1-aa61-1d5d83e99088 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.111647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 303/602] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:24.112454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905c04c-e108-4ac1-aa61-1d5d83e99088 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.113569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.114358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.115499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.115946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.117738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:24.118194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:24.119755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1cbcc-c896-4cb2-ac43-423ae8df4e19 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:24.136864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 300/603] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:24.138199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.138485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.138758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.145869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:24.161530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:24.162804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:24.247843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 301/604] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:24.248914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.250425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905c04c-e108-4ac1-aa61-1d5d83e99088 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.297655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.298660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.298937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.299203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.299938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:24.300921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:24.302564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:24.330676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 304/605] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:24.337007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.385451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:24.406286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65027223-a0d4-4564-8c01-68f842b65461 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.449483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.451814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.452239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.452652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.470130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:24.471850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:24.487224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:24.488319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:24.489429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:24.492035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 302/606] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:24.498094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.529008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:24.559808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31cef7-c341-40c7-a217-f5d2b9dbdb2e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.606229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.606802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.607078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.607348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/f236664fa09b4a4aafe6980768fc263b/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.630228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=f236664fa09b4a4aafe6980768fc263b, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:24.631954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:24.650835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 305/607] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:24 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/f236664fa09b4a4aafe6980768fc263b/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:16:24.655623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.729922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb37fd0-0e46-4d59-8701-ffe59a8a4195 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.795297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb32ff-7c03-43c4-8812-46a71f12c32e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:24.834751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.835445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.835756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.836208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.839756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:24.845024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:24.845781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:24.846159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:24.846532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:24.847338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:24.847776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:24.849346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:24.850121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:24.857449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:24.858718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:24.859829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 306/608] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:24.864477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.882146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 303/609] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:24.891064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:24.904085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:25.002412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edad53-fefb-47ef-9583-29e60e314bd1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:25.017733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fbc74-f165-4fc0-b695-a2030e806fa9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:25.042029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:25.043038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.043771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.044198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.045126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:25.045623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:25.047339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:25.052546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:25.053321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.053995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.053995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/86cd9a25b7ed4206a9b5939024168475/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.058380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 307/610] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:25.063766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:25.071263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=86cd9a25b7ed4206a9b5939024168475, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:25.072913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:25.095761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 304/611] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:24 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/86cd9a25b7ed4206a9b5939024168475/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:16:25.138489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:25.187960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5acd7f-8e55-47ea-af95-be5ce54a9685 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:25.233471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:25.234421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.234870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.235307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.253074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:25.254571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:25.279016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:25.280148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:25.281123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:25.281123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:25.281123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:25.281123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 308/612] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:25.288663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:25.356373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:25.421265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c6b5-d2ea-4bf0-9a94-a9d0c8f37d8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:25.489433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:25.490053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.490327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.490595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.491249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:25.491569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:25.493500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:25.511321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 305/613] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:25.516725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:25.666524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a9881-9eb7-48f7-95c1-3c990055f2ca None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:25.681608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:25.718822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:25.719470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.719773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.720042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.729302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:25.730762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:25.745529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:25.746556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:25.748504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 309/614] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:25.755818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:25.890437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034de330-a4a4-4fce-840a-c28941bb7490 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:25.908223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:25.928839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:25.929449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.930296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.930566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.931294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:25.931674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:25.933048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844acd0-f855-4d52-b08e-fde3ec09e48d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:25.951161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 306/615] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Sep 9 06:16:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:25.957027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:25.964916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e30b-95fe-4217-b195-6c4f8e407e3f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.965713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e30b-95fe-4217-b195-6c4f8e407e3f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.966192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e30b-95fe-4217-b195-6c4f8e407e3f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:25.968967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 307/616] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:25.974258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc5c64-5afd-4077-aefc-1f56e6c3a8b4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:25.974738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc5c64-5afd-4077-aefc-1f56e6c3a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:25.975726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc5c64-5afd-4077-aefc-1f56e6c3a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.014559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-97cc5c64-5afd-4077-aefc-1f56e6c3a8b4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:26.017116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-97cc5c64-5afd-4077-aefc-1f56e6c3a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:26.150981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 308/617] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:26.154461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dba2e2-f19c-4b89-a97b-2de68b3601a5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:26.182845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6ee3c-9977-4ccd-961a-5119b5fab474 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.183142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6ee3c-9977-4ccd-961a-5119b5fab474 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.183324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6ee3c-9977-4ccd-961a-5119b5fab474 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.186004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 309/618] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:26.189982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0330845-f510-496a-9c23-0571c47b738b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.190263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0330845-f510-496a-9c23-0571c47b738b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.190553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0330845-f510-496a-9c23-0571c47b738b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.204261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:26.204935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.205232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.205445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.209332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b0330845-f510-496a-9c23-0571c47b738b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:26.210530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b0330845-f510-496a-9c23-0571c47b738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:26.223126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:26.224593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:26.247334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:26.248341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:26.249320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:26.249320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:26.249320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:26.249320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 310/619] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:26.255652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.299961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:26.315827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 310/620] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:26.388700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34d1ca-976e-4d9c-8901-d9ba62d4408c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:26.437490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:26.438022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.438284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.438553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.439192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:26.439489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:26.440878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:26.456966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 311/621] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:26.462546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0330845-f510-496a-9c23-0571c47b738b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.495318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.512320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:26.602044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0330845-f510-496a-9c23-0571c47b738b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:26.643419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:26.663240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:26.663924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.664420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.665113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.666303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f043b-c904-4aa1-a1a4-e227991b9e59 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:26.672218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:26.673645 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:26.686594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:26.687628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:26.688566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 311/622] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:26.689104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.713543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:26.714183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.714465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.714901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.729401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:26.735385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:26.736358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:26.737224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 312/623] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:26.738564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.813750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:26.824184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46aab-63a2-4873-b8a0-f1a28638dc4e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:26.847249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336730ea-913d-42e3-ba0f-1b68b5eeadf5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:26.873492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:26.874277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.874663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.875038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.886619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:26.891824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:26.892963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:26.894037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:26.895245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:26.896051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:26.896228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 312/624] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:26.896695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:26.897187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:26.897297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.898231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:26.898839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:26.900514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:26.919817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 313/625] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:26.920912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:26.944055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.016418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f954-2c9c-4d32-bd63-d2a6f8e52b65 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.054117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.055698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.056203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.056467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.057289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:27.057724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.059317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.068627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ddbb3-9a33-42b6-a99d-ad11050e5b15 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.073494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 313/626] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:27.074487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.094535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.117329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.118690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.119122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.120140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.120983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:27.121440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.123460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.135537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 314/627] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:27.136588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.193093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756d6e2-e047-48c8-97cf-b3e7efe82833 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.226319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.230146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.231012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.231404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.231860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/cb474b7f08db4ff9b9fdf8eb8ca658af/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.250849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=cb474b7f08db4ff9b9fdf8eb8ca658af, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.253350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.275660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01620443-8fec-4776-a4ce-6492bff244df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.279911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 314/628] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:27 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/cb474b7f08db4ff9b9fdf8eb8ca658af/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:27.280934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.322436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.323620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.324105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.324627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.342610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.345117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:27.366529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:27.367955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:27.369078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:27.369078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:27.369078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:27.371428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 315/629] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:27.383401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.434970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.491945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b54076-5765-443d-b27e-fb9a2732c597 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.537111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.538084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.538563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.539033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.551634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.553029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.558693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f51d8c-38a0-409e-a0d5-49d089f4f2be None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6885761e-fb82-4e21-a664-ffea98688c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:27.573007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:27.574248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 315/630] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ed30-9d38-4945-8334-7a8b3e87e444 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ed30-9d38-4945-8334-7a8b3e87e444 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.575377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794ed30-9d38-4945-8334-7a8b3e87e444 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.576279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 316/631] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:27.582169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.582450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.582723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.589259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.607165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:27.608612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:27.610051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.610959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.611339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.612553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.613311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:27.613720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.615211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.632432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 316/632] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:27.634569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.697692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 317/633] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:27.699837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.773313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae07d2-34d8-4627-a3a4-53270b4b837d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.780551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.809965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.810581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.810862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.811136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.811803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:27.812109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.813488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235727f-933b-48fb-b947-b24a1480e4bc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.826278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 317/634] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:27.827326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d23bcf-8043-4257-8510-b525de0a3231 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.827661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d23bcf-8043-4257-8510-b525de0a3231 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.827939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d23bcf-8043-4257-8510-b525de0a3231 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.833314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 318/635] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:27.833314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d23bcf-8043-4257-8510-b525de0a3231 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:27.856478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04235-61b4-4b93-8e82-1f7b789f6611 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.870886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:27.909141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.911080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.911605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.912095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.922602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:27.924122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-066cb23e-a495-4ea3-af82-5c9b1d61a33a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:27.942018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:27.943045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:27.944068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 318/636] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:16:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:27.944718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.947136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.947136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:27.951770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d23bcf-8043-4257-8510-b525de0a3231 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:27.965081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:27.966220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:27.992078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:27.993334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:27.993957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:27.994466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.003750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.005420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:28.018374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:28.019347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 319/637] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:28.021059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.042415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 319/638] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:28.042890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.095754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.146814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.156483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ffcd2-7c49-4c94-9718-1008c97b87f7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.177322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd150a-fd10-42c7-a2c8-07c6177e3c16 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.201589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.202720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.203440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.203918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.205081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:28.205663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.208044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.223385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.224168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.224524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.224871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.231591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 320/639] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:28.232663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.238229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:28.244673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:28.245627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:28.246475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 320/640] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:28.247189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.342338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05614055-95e7-40d5-bc2b-8e2486984dfe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.367873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.372539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fed84-1472-450d-be69-e7a21263a746 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.380692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.381299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.381580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.381849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.383386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:28.383903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.385553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.409382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 321/641] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:28.409382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.412272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.412891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.413170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.413444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.425627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.481697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.483153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.518926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec419fe-e221-4e7f-9f9e-9fa54f30b6a4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.560839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.561456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.561733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.562000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.562662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:28.562961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.564355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.575141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 322/642] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:28.576251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.589708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.598736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 321/643] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:28.599358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.658178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.689831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f8fe4-917f-4c6d-9259-e86c49e60102 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.708239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfaaf8-0d8e-420a-95f7-d2f9fd0e2231 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.726707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.727663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.727946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.728217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.738873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:28.744991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:28.746029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 323/644] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:28.746901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.766616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.767229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.767509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.767813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.836603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.841014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.842493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.858022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d207-704a-4de0-99d1-8c6fd5f9772a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:28.892898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:28.893832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:28.894338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:28.894753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:28.896878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:28.907203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:28.908806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:28.923390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:28.924371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:28.925449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:28.925449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:28.925449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:28.925449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 324/645] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:28.925449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:28.957428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 322/646] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:28.958278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.037767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a4bd-2021-458d-bee8-543a7e49c9af None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.055660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.059826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4589ff-c39c-4204-848f-0076bb5718d4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.076648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.077275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.077668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.077930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.089994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.091600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.097878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.098488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.098767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.099034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.099722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:29.100030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.101929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:29.107897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:29.109091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:29.110214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:29.110214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:29.110214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:29.110829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 325/647] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:29.111680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.112770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 323/648] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:29.113717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.159116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.221802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9bcd1-0d3c-4acf-912b-ba9894445ba2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.224170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f895332-d30c-4360-8252-d2ba0e8e76bd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.256617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.257250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.258071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.258350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.261882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.262492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.262769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.263039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.263716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:29.264018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.265396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.269251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.270684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.279701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 326/649] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:29.280562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:29.289729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:29.290924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:29.291869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:29.291869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:29.291869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:29.291869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 324/650] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:29.291869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.306923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.393176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.397452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce5589-7e5d-4fbd-a7f1-a522e3124f31 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.400417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adb40d-ecc4-4303-8062-a22f8ca167a3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.435189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.435819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.436072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.436215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.436378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.436704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.436985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.437095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:29.437259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.437353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.438627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.449568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.455841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 327/651] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:29.455982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.456064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:29.470441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:29.471673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:29.472585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:29.472585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:29.472585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:29.472585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 325/652] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:29.472585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.539528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.580823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cd847-d7bb-415f-b9de-a3c3bddc7111 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.587189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca283-ad04-4b9e-a19e-8e8a4dbc80bc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.615446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.616127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.616409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.616681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.623424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.624072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.624352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.624624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.624979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.625273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:29.625574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.626421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.626926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:29.638456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:29.639440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:16:29.640434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:29.644370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 326/654] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:29.644483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 328/654] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:29.645048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.645144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.666085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.745131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f44e5d-2113-431a-803e-3152e7958c61 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.745705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29248878-d3c3-49dc-b67d-bdbc4280289c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.768649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.778749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.779363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.779665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.779938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.780405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.780576 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:29.780879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.781040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.781321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.781594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.782235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.788659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.790088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.793842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 329/655] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:29.794630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:29.801845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:29.802937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 327/656] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:29.804046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.875692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:29.898242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baab458-17ac-41ce-8632-a1a91baf0f96 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.902265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562975b1-8d61-44f5-8ffb-a1dd8420841e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:29.932899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.933497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.933772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.934038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.934749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:29.935049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.936073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:29.936520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e22ba-e851-4b69-b310-cf8d5fe7aa16 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.936701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.936981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.937255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.943796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:29.945229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:29.955164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 330/657] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:29.956457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f400-abcc-4c14-aa23-c22147fe7c56 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:29.956791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f400-abcc-4c14-aa23-c22147fe7c56 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:29.957114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f400-abcc-4c14-aa23-c22147fe7c56 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:29.957616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:29.958617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:29.964276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 328/658] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:29.964904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 331/659] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:29.964998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:29.965082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549f400-abcc-4c14-aa23-c22147fe7c56 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.009559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.065702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021207dc-424b-4dc7-af01-60087f6b8311 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.066563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549f400-abcc-4c14-aa23-c22147fe7c56 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.098719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.099347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.099482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.099655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.099929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.100023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.100245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.100516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.101694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:30.102012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.103654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.106068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.109102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.110549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.115805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 332/660] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:30.116619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:30.122223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:30.123208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:30.124216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 329/661] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:30.124905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.216789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327eb6d-580a-4f48-94cc-e89f8a09e7fb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.225757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6051373-98ea-4768-9755-a82f912cc66a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.229286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.250209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.250813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.251088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.251360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.261393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.261997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.262272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.262542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.263256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:30.263590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.264996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16afaec0-9b35-4a28-acd7-925a9d03c827 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.276878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 330/662] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:30.278040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.278322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.278592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.295527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:30.296724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:30.315055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.321830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.323268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.363082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 331/663] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:30.363946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.438217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 333/664] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:30.438918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.472662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.483247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8beaee-488e-4f79-9e35-83966183a648 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.520319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.520926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.521323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.521598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.522453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=dc56896d10094aed91db84ea2bf9dfb2) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:30.522765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.524909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.535666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 332/665] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/role_assignments?user.id=dc56896d10094aed91db84ea2bf9dfb2&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:30.535666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.550013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.559156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed04d1a-2336-44b2-93cb-e83c42b46c7a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.601798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.602483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.602763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.603036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.603828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:30.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.605615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.626740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 334/666] 10.222.0.25 () {56 vars in 1139 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:30.627715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.677863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fcba-0cf4-48a4-8a12-3c083132d231 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.716584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.717580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.717864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.718427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.719200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=0f8bf35f310b4b558dde597cde49655f) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:30.719511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.720998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.727584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 333/667] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/role_assignments?user.id=0f8bf35f310b4b558dde597cde49655f&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:30.728911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.746099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeb4a0-a8d6-4a85-8214-c96a22343b36 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.766489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.797147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.797808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.798087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.798360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.816130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:30.817790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:30.839075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:30.840114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:30.841389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:30.841389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:30.841389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:30.841389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 335/668] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:30.841389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.851984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:30.873681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dfd36-5876-43dc-9d07-3a980781bca7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:30.916393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:30.917093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:30.917371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:30.917639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:30.932715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:30.938943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:30.939995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:30.940846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 334/669] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:30.941567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:30.984288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01595-e6b3-4c97-8362-238441da7041 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.026767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.027677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.027960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.028235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.043100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.067243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28103-33fc-4422-b93d-2b09d3123d4a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.113446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.114305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.114906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.115452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.116040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.116540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.119123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:31.119692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.121313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.124334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.141403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 335/670] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:31.142552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.237423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 336/671] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:31.238686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.262580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee1afb-bc89-47ae-ad78-b16922c64ce5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.299312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.300106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.300344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.300628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.301284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:31.301594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.302960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.317139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 336/672] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:31.318033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.323851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.348046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d9ffc-ab96-434f-ae60-51e20aa2fa11 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.356187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.385524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.386315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.386717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.387092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.394511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.396110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:31.411299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:31.412593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:31.414585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:31.415957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 337/673] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:31.416747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.436163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94af03-995d-435a-86cf-d13d211895f7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.479468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.480625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.481048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.481686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.497409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.499610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:31.520020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:31.521197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:31.522327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:31.522327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:31.522327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:31.529485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 337/674] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:31.529485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.534927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13033a-3c0d-4e1c-b5ca-20cf93bfa1be None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.578401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.579561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.580529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.580961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.594646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.596650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.598282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:31.618238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:31.619430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:31.620547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:31.620547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:31.620547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:31.622295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 338/675] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:31.623286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.646925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.650318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ce97-05ef-46ad-9a08-8c2b41b3a81a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.693893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.696009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.697135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.697558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.698701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:31.699467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.701761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.719269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 338/676] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:31.721053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.744127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e919f-4c23-4714-b7f1-a2643145dd48 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.781204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.781735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.782115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.782527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.783277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:31.783720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.785194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.802946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 339/677] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:31.804131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.857327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce966900-25e9-47e4-b62a-58b9c8af8fe0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.881682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.899360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:31.903159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.904663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.905239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.905733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.906609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:31.907125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.909050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.922558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 339/678] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:31.923901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:31.926074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e0698-c44f-43a8-894c-f8b4cda32d59 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:31.963000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:31.963641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:31.963928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:31.964211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:31.964927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=f236664fa09b4a4aafe6980768fc263b) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:31.965236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:31.966665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:31.974606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 340/679] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/role_assignments?user.id=f236664fa09b4a4aafe6980768fc263b&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:31.975483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:32.083038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c816b25-3a28-458e-9272-0e623d99f3dc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:32.088421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7198dac-78f1-4747-9905-69c0fcc4dc5c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:32.130541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:32.130541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:32.130541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:32.130541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:32.138204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:32.139813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:32.155057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:32.156139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:32.157398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 341/680] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:32.161915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:32.167029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:32.167676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:32.167786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:32.167997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:32.168275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:32.184368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:32.190909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:32.219723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:32.220783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:32.222681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 340/681] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:32.232555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:32.306272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:32.343871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cea86-3184-45b9-9cc5-5bdbc09e03b0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:32.383963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:32.384689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:32.384975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:32.385252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:32.385988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:32.386298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:32.388010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:32.393680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dcac2-abaa-4cb4-a452-d34c9b74ce43 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:32.409802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 342/682] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Sep 9 06:16:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:32.415802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:32.446826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:32.447441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:32.450072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:32.450355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:32.451059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:32.451363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:32.454725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:32.483342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 341/683] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:32.489729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:32.539879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:32.598048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99012ac0-6712-4cda-978c-eb0b72455f9a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:32.635484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:32.636500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:32.636982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:32.637454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:32.713490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:32.715450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:32.737660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:32.752596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85795ce7-1e50-44a4-86b6-9d533c7a4e63 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:32.779089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:32.826208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 343/684] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:32.837134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:32.844227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:32.844912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:32.845200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:32.845477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:32.846216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=86cd9a25b7ed4206a9b5939024168475) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:32.846525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:32.848010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:32.856234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 342/685] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:32 2022] GET /identity/v3/role_assignments?user.id=86cd9a25b7ed4206a9b5939024168475&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:33.018261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9bb4e-b345-4796-ad71-793830d9ba32 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:33.073684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:33.074742 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.075231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.075762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.090915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:33.092759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5fcb3aac-e2ec-45e4-a7b6-68bcb1660a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:33.110590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:33.111599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:33.112560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:33.112560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:33.112560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:33.114626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 344/686] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:33.126423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:33.191281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd812e-3b9d-47aa-a86f-9efa8c7e47f2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.191608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd812e-3b9d-47aa-a86f-9efa8c7e47f2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.191862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd812e-3b9d-47aa-a86f-9efa8c7e47f2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.193437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 345/687] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:33.199100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c280e-b5b0-418f-9ba1-3becef97b281 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.199381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c280e-b5b0-418f-9ba1-3becef97b281 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.199718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c280e-b5b0-418f-9ba1-3becef97b281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.222294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0a3c280e-b5b0-418f-9ba1-3becef97b281 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:33.223482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0a3c280e-b5b0-418f-9ba1-3becef97b281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:33.232152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:33.262991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235391-75fa-4582-a518-f58f3a392fc5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:33.312012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:33.312676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.313064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.313336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.314101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:33.314404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:33.315800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95d861-7d8f-4392-ab50-192ba283aeea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:33.318628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 346/688] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:33.327579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df128a1-7983-4705-963c-458b9a1e4127 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.328518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df128a1-7983-4705-963c-458b9a1e4127 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.329218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df128a1-7983-4705-963c-458b9a1e4127 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.331155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 347/689] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:33.336364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 343/690] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:33.337453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df128a1-7983-4705-963c-458b9a1e4127 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:33.342240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.342486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.342813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.361913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:33.363077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:33.383593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:33.444703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 344/691] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:33.490092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df128a1-7983-4705-963c-458b9a1e4127 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:33.526286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:33.526920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.527202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.527475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.534452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:33.535941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:33.549094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:33.550298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:33.551199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 348/692] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:33.555983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:33.681172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9273b69-bf60-4546-9ae4-482c91e797b3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:33.717899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:33.722948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:33.723617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.723882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:33.724490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.724759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.725460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:33.725763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:33.728008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:33.740604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 345/693] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:33.747248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:33.760747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:33.857616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb79185-b9b2-4cb0-a06b-2765e875a30f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:33.897277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8de20a-97f7-44a2-ba56-7497f13d4ef7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:33.925366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:33.925979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.926258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.926532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.927269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=cb474b7f08db4ff9b9fdf8eb8ca658af) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:33.927604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:33.929040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:33.935083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:33.935806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:33.935941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 346/694] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:33 2022] GET /identity/v3/role_assignments?user.id=cb474b7f08db4ff9b9fdf8eb8ca658af&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:33.936035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:33.936270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:33.936762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:33.977266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:33.991625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:33.992898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:33.993771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:33.996413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 349/695] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:34.003680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:34.039285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:34.090050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d727-2283-4e50-976a-857ab563d564 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:34.126911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:34.165972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:34.166627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.166898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.167163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.181276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:34.182822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:34.190449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2afa95-4a64-476e-8dbb-1bf26e6b230d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:34.202154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:34.203161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:34.204356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:34.204356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:34.204356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:34.204356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 347/696] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:34.211265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:34.239263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:34.239913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.240194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.240477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.241137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:34.241441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:34.243784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:34.252722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 350/697] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:34.259226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:34.355758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc4be8-999c-434c-a375-9714ccb4921e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:34.388229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a6d24-daac-4093-8295-aef2b00f7229 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:34.391048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:34.409274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:34.410261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.411243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.411890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.412842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:34.414108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:34.416249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:34.423536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:34.437505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 348/698] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:34.443023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:34.443646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.443916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:34.444044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.444240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.458236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:34.460001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:34.478940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:34.479951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:34.480863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:34.480863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:34.480863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:34.480863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 351/699] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:34.492951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:34.575604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17cdd-8c26-4b58-aeec-cc1d17444a5d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:34.612277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:34.612945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.613223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.613492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.614191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:34.615643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:34.648272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 349/700] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:16:34 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:34.717963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b90676-4661-4db6-8383-0011d8b9fb1b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:34.766864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:34.767921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.768387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.768872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.769574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:34.769879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:34.771244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f289-0cb5-4577-ae65-6fafc1c90414 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:34.790267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 352/701] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:34.796542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:34.800206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a6f9-3cff-4269-9206-4727f9ec7e03 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.800668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a6f9-3cff-4269-9206-4727f9ec7e03 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.801297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a6f9-3cff-4269-9206-4727f9ec7e03 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.804640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 353/702] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:34.816883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:34.817191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:34.817558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:34.854552 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:34.855789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:34.898793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:34.919840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:34.969545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 354/703] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:34.979009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192792e-596c-46eb-80e4-a45de809a495 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:35.014667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:35.015678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.016137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.016588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.026062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:35.027765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:35.040292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:35.041430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:35.042421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:35.050733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:35.051758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 350/704] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:16:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:35.168311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99516ad3-95fd-475c-a71a-3210ce144f93 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:35.205798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:35.206413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.206691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.206963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.207741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:35.208103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:35.209537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:35.234956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 355/705] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:16:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:35.241008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:35.276006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:35.313964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:35.420972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce73db6-aa75-4494-bc7e-6eb3af35b010 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:35.431681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042aae13-4521-49c2-85d7-cfc1443c6c6e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:35.433426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:35.457609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:35.458250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.458527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.458796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.459383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:35.460912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:35.472017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:35.473051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:35.473902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:35.478456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 356/706] 10.222.0.25 () {56 vars in 1129 bytes} [Fri Sep 9 06:16:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:35.480972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:35.524754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d39782-dedf-42b9-a9fa-e57264f8ffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:35.525632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d39782-dedf-42b9-a9fa-e57264f8ffcb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.525956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d39782-dedf-42b9-a9fa-e57264f8ffcb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.526229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d39782-dedf-42b9-a9fa-e57264f8ffcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.603993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:35.618473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180cd5a-162c-4951-9621-3d5046459ed5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:35.661394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:35.662488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.662773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.663046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.664032 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:35.664359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:35.666455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:35.675894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 357/707] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Sep 9 06:16:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:35.677161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:35.713882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d39782-dedf-42b9-a9fa-e57264f8ffcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:35.715357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d39782-dedf-42b9-a9fa-e57264f8ffcb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:35.788513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab31819-c8ed-4346-a2c3-649ca611019c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:35.825284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:35.825899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.826175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.826448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.827235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:35.827556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:35.829298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023b66-c773-431b-893c-2856de4cfb88 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:35.832128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 351/708] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:35.837221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b935d39-af7e-4526-bc27-abcadb88e1d6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.837531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b935d39-af7e-4526-bc27-abcadb88e1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.837807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b935d39-af7e-4526-bc27-abcadb88e1d6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.839398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 352/710] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:35.839630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 358/710] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Sep 9 06:16:35 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:35.842029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b935d39-af7e-4526-bc27-abcadb88e1d6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:35.845361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:35.846781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:35.847156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:35.847447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:35.888809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:35.890199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:35.913018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:36.009134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b935d39-af7e-4526-bc27-abcadb88e1d6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.041367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 359/711] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:36.065939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.066626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.066909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.067175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.079561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.081014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:36.096892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:36.099041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:36.101767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:36.101767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:36.101767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:36.101767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 353/712] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:36.107566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.224772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf10d8-58be-4a38-a764-29b8c73b1518 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.262495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.263107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.263381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.263694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.264333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:36.264629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.266003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.282683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 360/713] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:36.288394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.345455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.360447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:36.363194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:36.410890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5210-3ebe-47be-90d6-712ed38b0566 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.448639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.449599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.450042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.450453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.458341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.459968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:36.473723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:36.474657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:36.475593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:36.478534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 354/714] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:36.488487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.555581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5107-9299-4b47-9872-f37f8fc1621c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.620112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67853ce8-1917-4610-b80a-977f2a805ec7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.624491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.625497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.625976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.626449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] PATH_INFO: `/v3/services/network` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.627253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.628955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:36.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:36.640708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:36.641543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:36.644483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 361/715] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/services/network => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:36.650453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:36.651942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.689736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.690410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.690693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.690966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.691699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:36.692010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.693704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.717244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 355/716] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:36.723370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.730695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:36.764786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636157a9-226f-41fd-972f-d7ebcb749c5e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.801701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.803194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.803836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.804522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.805385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:36.805916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.807629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7923f2-a7a7-476e-904b-2070faa966b6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.820077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 362/717] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:36.822937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba5a78-26fc-4bfe-887b-dc611813905c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.823477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba5a78-26fc-4bfe-887b-dc611813905c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.823945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba5a78-26fc-4bfe-887b-dc611813905c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.825890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 363/718] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:36.827529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba5a78-26fc-4bfe-887b-dc611813905c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:36.848672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46476d-dbe9-4e06-87fd-0572f32aa02b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:36.898092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:36.898994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.899773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.900632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.901541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=dc56896d10094aed91db84ea2bf9dfb2) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:36.902147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:36.904480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae122433-acd2-4c4d-b484-256dc84ce058 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:36.917565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 356/719] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/role_assignments?user.id=dc56896d10094aed91db84ea2bf9dfb2&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:36.917565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:36.917565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:36.917565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:36.956054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:36.957295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:36.986884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba5a78-26fc-4bfe-887b-dc611813905c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:37.028873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:37.032141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:37.032767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:37.033740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:37.034021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:37.034667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] RBAC: Adding query filter params (type=network) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:37.034971 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:37.036428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:37.044818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:37.051688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 364/720] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:16:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:37.091813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 357/721] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:37.424187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:37.553685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5655cdc-46d1-4f55-adbd-96fd87830b8d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:37.606996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f2fb8-3bed-43b0-a276-a6a0953aee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:37.607681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f2fb8-3bed-43b0-a276-a6a0953aee88 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:37.607963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f2fb8-3bed-43b0-a276-a6a0953aee88 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:37.608234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f2fb8-3bed-43b0-a276-a6a0953aee88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:37.631323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f2fb8-3bed-43b0-a276-a6a0953aee88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8a4f2fb8-3bed-43b0-a276-a6a0953aee88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:37.637362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:37.638332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:37.639817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 365/722] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:37.644301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:37.729693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd60e9-55a0-49ee-aed3-f1e3e80511e4 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:37.730323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd60e9-55a0-49ee-aed3-f1e3e80511e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:37.730799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd60e9-55a0-49ee-aed3-f1e3e80511e4 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:37.733670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 366/723] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:37.737933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:37.738444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:37.738943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:37.759292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:37.760876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:37.806574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92f61d-f2b4-46e7-8847-727a6efcd321 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:37.855582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:37.862275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:37.862896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:37.863176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:37.863448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:37.864171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:37.864476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:37.865849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:37.882354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 358/724] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:37.886933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 367/725] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 06:16:37.891582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:37.944114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:38.021357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9fa01-f9a3-419d-b749-18da6f77570b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:38.057788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:38.058423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.058701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.058977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.070183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:38.071639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:38.087107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:38.088103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:38.089015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:38.089015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:38.089015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:38.089015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 359/726] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:38.094955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.219037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.238725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4337010-0b00-423d-8a27-51239ca98519 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:38.295586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:38.296131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.296408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.296679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.297328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:38.297629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:38.299016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:38.319376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 368/727] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:38.327925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.371152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:38.407593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d88f84-792b-40fb-aeb8-688d19867cbe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:38.418258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:38.471472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:38.472136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.472416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.472701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.491892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:38.497224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:38.498136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:38.498983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 360/728] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:38.506736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.513861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0baeea-d06c-4a7f-be53-9a7006acc003 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:38.577274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:38.577921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.578198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.578472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.590734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:38.592217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:38.611876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:38.612912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:38.613913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:38.614560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 369/729] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:38.623358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.695414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8ce1b-b5be-4a82-ae87-520bbb453200 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:38.756998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:38.758033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.758487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.758939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.759814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:38.760276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:38.762492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:38.777462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 361/730] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:38.785132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.795888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:38.815706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ad234-5db1-4b3b-bef7-82e816793fc2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:38.833110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:38.885992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:38.886635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.886876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.887146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.887910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:38.888224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:38.889655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99b092-4d72-430d-9acc-db86619702d3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:38.915218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 370/731] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:38.916869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593257dc-9227-4be7-a815-3e8d03f8f85f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:38.917357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593257dc-9227-4be7-a815-3e8d03f8f85f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:38.917829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593257dc-9227-4be7-a815-3e8d03f8f85f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:38.923403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 371/732] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:38.924497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593257dc-9227-4be7-a815-3e8d03f8f85f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:38.966764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c120-1c7c-4e6d-91f0-f351bf1dcab7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.002950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.003997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.004475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.004963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.022588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.025482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-25a1d2f8-b179-449d-bbe6-74e6420e9d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:39.045077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:39.046138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:39.047031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:39.047031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:39.047031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:39.048314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593257dc-9227-4be7-a815-3e8d03f8f85f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.050998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 362/733] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:39.053377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.053662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.053974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.077991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:39.079720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:39.090069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.090958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.091685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.092169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.131218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:39.149031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:39.173437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.178917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 363/734] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:39.178917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.181116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.295218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa0311-1c64-4fe8-a70f-ffa5e345d998 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.340904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.341443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.341731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.342006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.342662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:39.342968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.344371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.358013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 364/735] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:39.363275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.379054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 372/736] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:39.384787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.419917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:39.471401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:39.490784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5677-3768-4cd4-bf62-87e37496ae88 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.508406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de226a-2eed-4deb-a4a6-34a504d2ceab None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.543620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.544267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.544549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.544828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.545303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.545998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.546361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.546706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.554790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.556862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.557994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.559503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5a41b10f-461d-40bf-9c2f-d587a5d81de7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:39.571211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:39.572430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:39.573575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 365/737] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:16:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:39.574182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89b554-1135-4dac-a5ec-aa1c699812e0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.574182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89b554-1135-4dac-a5ec-aa1c699812e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.575850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89b554-1135-4dac-a5ec-aa1c699812e0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:39.575958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:39.576972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:39.578038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:39.578038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:39.578038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:39.578248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 366/738] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:39.578248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89b554-1135-4dac-a5ec-aa1c699812e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.580882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 373/739] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:39.581740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.692189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89b554-1135-4dac-a5ec-aa1c699812e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.697297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0868dc-b46c-48e3-be17-8b333915819a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.701827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:39.716112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:39.733342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d024e5-9947-4d86-aaa5-22ccc4a04423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.733950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d024e5-9947-4d86-aaa5-22ccc4a04423 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.734228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d024e5-9947-4d86-aaa5-22ccc4a04423 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.734496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d024e5-9947-4d86-aaa5-22ccc4a04423 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.734597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.735145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d024e5-9947-4d86-aaa5-22ccc4a04423 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.735236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.735489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.735795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.736524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:39.736620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d024e5-9947-4d86-aaa5-22ccc4a04423 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.736832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.738238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.751770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 367/740] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:16:39 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:39.753367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5a1c-e559-437d-8e79-fa52298138d7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.753996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5a1c-e559-437d-8e79-fa52298138d7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.753996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f5a1c-e559-437d-8e79-fa52298138d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.759598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 374/741] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Sep 9 06:16:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:39.759598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.777634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-556f5a1c-e559-437d-8e79-fa52298138d7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:39.778862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-556f5a1c-e559-437d-8e79-fa52298138d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:39.867582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 368/742] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:39.867582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3a14-1495-450d-82b7-b1bcd0f7c913 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.867582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3a14-1495-450d-82b7-b1bcd0f7c913 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.867582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3a14-1495-450d-82b7-b1bcd0f7c913 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.872325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 369/743] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:39.873603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3a14-1495-450d-82b7-b1bcd0f7c913 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:39.881450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac51a8-55f2-42a1-85ba-df496abb1e27 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:39.919896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:39.920563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.920785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.921057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.921705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:39.922011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:39.923918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315cf1b-2bb6-47cd-a341-0d47d7b0157e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:39.939750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 375/744] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:39.941140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:39.941424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:39.941695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:39.967374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:39.969194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:40.001196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.004953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.017243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3a14-1495-450d-82b7-b1bcd0f7c913 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.056538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.058197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.058721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.059301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.061633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 376/745] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 06:16:40.062528 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.154641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.156696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.182106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d890e8-363a-48e9-9031-6ec3b5cfcbb2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.218563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.219223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.219498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.219911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.228521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.230042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:40.242734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:40.243748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:40.244751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:40.248767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 377/746] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:40.249609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.274862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.283891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.287930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 370/747] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:39 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 06:16:40.289753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.361050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e88bdc-16c8-442f-9651-a870238e3ef5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.405389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.406038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.406727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.407006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.419779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:40.421518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344d36-cd33-45e5-9b3f-5d40bbd8a1b1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:40.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:40.425654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:40.426508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 378/748] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:40.427204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.459467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.460278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.460618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.460887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.461578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:40.461883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.463463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.481166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 371/749] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:40.481779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.516740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.535204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e425cd8-742e-4cf1-98d1-e4895baa8288 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.535413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.573402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.574035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.574311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.574584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.585062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.586972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.592929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fdfbe-ef91-4a2f-8cd6-06f1dbc5e841 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:40.602922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:40.604139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:40.605265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:40.605265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:40.605265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:40.605265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 379/750] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:40.605265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.636792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.637265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.637541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.637906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.638458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.639911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.655629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 372/751] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:16:40 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:40.662479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.726535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b453-6acf-47df-bef8-5d1232bac284 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.781108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.781303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.781551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.781718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.782385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:40.782685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.784137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.785958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.795317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 380/752] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:40.796479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.831918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be1c2e-dccc-4ecc-b765-93e1dacfade1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.849256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:40.878670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.879703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.880210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.880686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.881614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=cb474b7f08db4ff9b9fdf8eb8ca658af) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:40.882158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.883847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:40.895575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 373/753] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/role_assignments?user.id=cb474b7f08db4ff9b9fdf8eb8ca658af&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:40.895575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:40.939276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76f725-7992-4b08-8716-59938b9c5625 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:40.982035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:40.982743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:40.983020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:40.983291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:40.983965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:40.984264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:40.985664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.003671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 381/754] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:41.012372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:41.040573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644cf8c-aeca-4e92-99c9-4a2f4f981d16 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.083194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.083923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.084217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.084485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.102339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:41.104174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:41.121989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:41.123001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:41.124018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:41.124018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:41.124018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:41.126195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 374/755] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:41.131279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:41.136072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:41.186554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:41.220316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41738735-81b2-4320-9ceb-8868ed3097d3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.284572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.285596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.286100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.286592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.297165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:41.302825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:41.322331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:41.323407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:41.324694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:41.330620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb948a-dc6a-403c-8035-70235d9d65c1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.334185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 382/756] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:41.340017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:41.403173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.404190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.404696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.405170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.406044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:41.406551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:41.408159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.426526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 375/757] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:41.431043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:41.543770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772048bf-a65d-46e1-b33c-f8580cc12460 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.555879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:41.577662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071d159-660f-486f-a376-a94b005c6d4e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.587885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:41.619855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.620475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.620841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.621130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.621831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:41.622137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:41.623605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.643843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 383/758] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:41.649685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:41.661460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.662152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.662804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.663193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.679941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:41.681886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:41.706327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:41.707387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:41.708404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:41.714337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 376/759] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:16:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:41.718997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:41.842736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcbadd-e8ad-4c20-b694-b2cbee21b26f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.915681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.916734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.917355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.917833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.918792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=e0acbc7456fe40a1b0d70c0a35976fae) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:41.919973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:41.921663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:41.934243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17295cac-ee9f-4c6f-80a3-225416992178 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:41.937887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 384/760] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:41 2022] GET /identity/v3/role_assignments?user.id=e0acbc7456fe40a1b0d70c0a35976fae&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:41.995471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:41.996607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:41.997100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:41.997610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:41.998597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:41.999148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:42.001005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524e5a0-a08a-4fee-8b37-fe10e7707dcc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:42.026456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 377/761] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:16:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:42.035322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:42.101631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:42.130297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05ef96-af5c-455f-a232-402f2b5a154d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:42.130836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05ef96-af5c-455f-a232-402f2b5a154d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:42.130836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05ef96-af5c-455f-a232-402f2b5a154d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:42.132952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 378/762] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:42.148713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:42.149033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:42.149289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:42.165664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:42.176116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:42.177381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:42.235317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a88dd-8f20-4cb1-a134-e779cbacf486 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:42.294595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 379/763] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:42.316184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:42.317017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:42.317296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:42.317566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:42.332140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:42.333615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:42.353858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:42.354812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:42.355737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:42.355737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:42.355737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:42.355737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 385/764] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:42.360697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:42.474109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a30f0-c52c-4dce-803a-83a607a4ccd6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:42.515066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:42.515713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:42.515994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:42.516267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:42.516975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:42.517278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:42.518707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:42.521211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:42.538743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 380/765] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:42.546198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:42.679168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:42.693526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b0245-17da-4ade-a8bb-378a348ea352 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:42.737759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:42.740539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:42.741303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:42.741714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:42.742001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:42.756654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:42.759078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:42.774508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:42.776557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:42.778716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 386/766] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:42.783586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:42.858563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181319f-1e78-4027-8154-67926b0b47e3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:42.930846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:42.931409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:42.931718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:42.931989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:42.932584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:42.933970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:42.964649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:42.966984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:42.968996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 381/767] 10.222.0.25 () {56 vars in 1129 bytes} [Fri Sep 9 06:16:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:42.975036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:42.995013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6c7c-b49b-4d2c-9765-4c05006d5afc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:43.076254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:43.077661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.078153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.078874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.079803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:43.080394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:43.082055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:43.100855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:43.108122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30db9f-594e-4f5d-b6a8-94c42d9b9a76 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:43.118446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 387/768] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:43.128483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:43.145111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:43.145605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.145605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.145911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.146493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:43.146800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:43.148735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:43.154167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 382/769] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Sep 9 06:16:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:43.163709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:43.163927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:43.338596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc117d-4723-4391-88b5-dac2237ec135 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:43.349564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7163db8-0d4f-42e2-855f-186f097447aa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:43.365235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:43.378921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:43.379576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.380056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.380558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.381510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:43.382513 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:43.388547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd968727-4de8-4184-ac6d-515da88a30cc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:43.394355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 383/770] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Sep 9 06:16:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:43.396076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa50b-d109-415c-9bcf-f937c510ae3a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.396368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa50b-d109-415c-9bcf-f937c510ae3a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.396648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa50b-d109-415c-9bcf-f937c510ae3a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.398380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 384/771] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:43.404593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa50b-d109-415c-9bcf-f937c510ae3a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:43.411920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27037d4f-3604-4a7f-98e4-882387a126a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:43.413405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27037d4f-3604-4a7f-98e4-882387a126a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.413854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27037d4f-3604-4a7f-98e4-882387a126a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.414415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27037d4f-3604-4a7f-98e4-882387a126a7 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/dc56896d10094aed91db84ea2bf9dfb2/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.418572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27037d4f-3604-4a7f-98e4-882387a126a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=dc56896d10094aed91db84ea2bf9dfb2, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:43.421491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27037d4f-3604-4a7f-98e4-882387a126a7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:43.439645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 388/772] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:43 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/dc56896d10094aed91db84ea2bf9dfb2/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:43.441777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d9c5e-2919-4306-8c47-9b01ead19b12 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.442071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d9c5e-2919-4306-8c47-9b01ead19b12 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.442573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d9c5e-2919-4306-8c47-9b01ead19b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.493599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-697d9c5e-2919-4306-8c47-9b01ead19b12 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:43.497614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-697d9c5e-2919-4306-8c47-9b01ead19b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:43.589392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa50b-d109-415c-9bcf-f937c510ae3a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:43.613748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 389/773] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:43.615184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd8d08-1453-401a-b83d-b208dce4f737 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.615523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd8d08-1453-401a-b83d-b208dce4f737 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.615877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd8d08-1453-401a-b83d-b208dce4f737 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.622088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 390/774] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:43.624016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.624597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.625057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.635929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:43.636219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:43.636901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.637176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.637502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.638228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] RBAC: Adding query filter params (interface=public, service_id=237a70e0e75c4649907b3a71b405a361, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:43.638552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:43.640092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:43.654693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 385/775] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:16:43 2022] GET /identity/v3/endpoints?service_id=237a70e0e75c4649907b3a71b405a361&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:43.656963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:43.667767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:43.676115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:43.734341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:43.801673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad7aee-16c1-4c44-8081-8ef39c1e15c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:43.888474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:43.889113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:43.889476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:43.889811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:43.890509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:43.891997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:43.902061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 386/776] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:16:43 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:43.910418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 391/777] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:44.002632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.175686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06776e-b18d-4558-9afa-23432ac097cb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:44.200502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:44.227382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:44.228450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.228930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.229428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] PATH_INFO: `/v3/services/network` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.230214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:44.231817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:44.246005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:44.247099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:44.248011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:44.250852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 387/778] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:44.259485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.280480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.427010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae907eb-02ca-429a-888f-8910b76afc69 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:44.464450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:44.466815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51051e4d-43bb-45c4-8250-1fb98a78e46f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:44.473193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:44.479010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.479354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.479682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.480317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] RBAC: Adding query filter params (name=network) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:44.480632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:44.482607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:44.493926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 392/779] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:44.498535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.544939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:44.545597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.545884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.546166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.579533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:44.586977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:44.587993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:44.588836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:44.591424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 388/780] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:44.597549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.672595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:44.687262 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8434c63-cabd-4fd4-b6f1-e4ef44138b9a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:44.737786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:44.738760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.739298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.740469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.741679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] RBAC: Adding query filter params (type=network) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:44.742194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:44.744052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da0a78-e175-4ec9-a61c-b47c34977929 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:44.749889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 393/781] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:44.751366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af645f5d-552a-411e-a5d4-7b5e203506bb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.753252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af645f5d-552a-411e-a5d4-7b5e203506bb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.753551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af645f5d-552a-411e-a5d4-7b5e203506bb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.756347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 394/782] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:44.757307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af645f5d-552a-411e-a5d4-7b5e203506bb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.779701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5244-46c2-48c3-84c7-ffc876da7adb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:44.816411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:44.817022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.817329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.817676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.818347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:44.818652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:44.820397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8c8d-82a3-4722-8e9b-37ebf2e8b45a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:44.824063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:44.831294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 389/783] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:44.832708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.833010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.833265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.854407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:44.855881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:44.876496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af645f5d-552a-411e-a5d4-7b5e203506bb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:44.916625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:44.917248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:44.917527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:44.917801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:44.918502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] RBAC: Adding query filter params (interface=public, service_id=915d75b09d9d4aa39cd58403ccc477fc, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:44.918808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:44.920196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:44.930162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 395/784] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/endpoints?service_id=915d75b09d9d4aa39cd58403ccc477fc&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:44.931395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.938876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 390/785] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:44.939931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:44.995309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:45.039780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702cb56-2cd2-4a09-8151-2359ddc53d32 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.065308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:45.078502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.079397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.079709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.079987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.090758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:45.092246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:45.108119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:45.110268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:45.112317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:45.112317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:45.112317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:45.112317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 396/786] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:45.116858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.120794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17c994-8169-4c0d-85d0-e3368aaa449e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.192812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.193582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.193740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.194505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.195210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:45.196909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:45.209772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 391/787] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:16:44 2022] GET /identity/v3/services => generated 772 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:45.232273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239f078-1d92-44e5-817d-98c1e2c80b2b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.270112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.270898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.271076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.271256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.271938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:45.272495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:45.273843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:45.286809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 397/788] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:45.292587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.341748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.403939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:45.466029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1efdc-5464-4413-911b-0eb87bf420a9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.481010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:45.487654 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14191c-7550-4461-b0f6-6d676d938032 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.511496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.513086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.513579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.514085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.528279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:45.530027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.530668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.530946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.531216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:45.536195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:45.537149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:45.538053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:45.540419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:45.540804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 398/789] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:45.542030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:45.555113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:45.568421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:45.569720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:45.570915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:45.574929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 392/790] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:45.580733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.719756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:45.729803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0167eb-e434-4f3d-9dda-83d302a17dde None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.763292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b2070-f4a0-4834-876f-96736e45775c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:45.789858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.790471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.790749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.791022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.791712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:45.792018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:45.793387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:45.814345 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 399/791] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:45.820883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.838167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:45.838796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:45.839076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:45.839349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:45.842369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:45.842694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:45.845810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:45.883760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 393/792] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:45.889519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:45.919337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:45.981568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89dbb6-5ccb-4250-ae01-56a94914b13f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.020827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.021465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.021743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.027575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.048275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.049743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:46.069926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:46.070991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:46.071959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:46.071959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:46.071959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:46.071959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 400/793] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:46.077783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.094614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:46.129713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3567ce4-53ac-4be5-a9d8-be10e4a5dc5a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.192675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345f8b7-d8dc-4974-9473-c52e4d012ee0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.218820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.219500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.219778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.220054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.232625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.233433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.233698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.234152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.234618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.234895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.235859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:46.236360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.238083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.253130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 401/794] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-567113ce-e9c8-4a21-8cf8-b28921c622d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:46.253307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:46.254221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:46.255135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:46.255135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:46.255135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:46.255135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 394/795] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:46.259706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c11f9-7645-4e42-8cad-04291965a025 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.259706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c11f9-7645-4e42-8cad-04291965a025 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.259706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c11f9-7645-4e42-8cad-04291965a025 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.260927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.261962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 395/796] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:46.263491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.263814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.264172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.288065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:46.289555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:46.349514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:46.365220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:46.375467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 396/797] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:46.376544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.407408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbf6a6-5398-459a-8d47-32352a07d715 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.472978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.473598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.473999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.474409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.475066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:46.475937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.477306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.489962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 402/798] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:46.500685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.555572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4693c66-cdc6-4aac-a49e-716ba033d358 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.648753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c817e-a36d-48c7-a065-334628b40e4d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.664409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.665072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.665365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.665642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.677873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.680918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.681621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:46.688177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.688866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.689198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.689478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.697163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.698594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5fc63c65-26ee-47cc-a3d4-240a1ef7575c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:46.700694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:46.701650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:46.702676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:46.710037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 397/799] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:46.710037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0dca0-0a54-460c-9fa8-b0198ddca47f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.710331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0dca0-0a54-460c-9fa8-b0198ddca47f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.710490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0dca0-0a54-460c-9fa8-b0198ddca47f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.712073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 398/800] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:46.712800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0dca0-0a54-460c-9fa8-b0198ddca47f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:46.717656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:46.718615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:46.719516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:46.722846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 403/801] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:46.723697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.773786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:46.818554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0dca0-0a54-460c-9fa8-b0198ddca47f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.837036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32194127-9657-4ede-bc22-e68942d3652a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:46.853114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.853730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.854010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.854285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.855019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:46.855330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:46.856835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeeac64-93a5-4c13-a59e-edb0fbadc279 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:46.874047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:46.874674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.874946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.875213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.876114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 399/802] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:46.877350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:46.877668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:46.877911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:46.891996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:46.896358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:46.898208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:46.898380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:46.899372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:46.900562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 404/803] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:46.901274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:46.963806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:46.974686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 400/804] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:46.976429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.026944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5067fd-3282-4305-90af-a37d94150745 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.037447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.070382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.070992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.071270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.071560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.072254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:47.072556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.073972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.093377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 405/805] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:47.094378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.108207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5664fb-bf3a-46c7-9dab-9cf38c3de069 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.143628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.144268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.144553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.144829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.155668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.157110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:47.172979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:47.173979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:47.174885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:47.174885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:47.174885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:47.179039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 401/806] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:47.179916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.213469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194c8e9-e24f-4758-821b-4e2f8d24d2da None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.224964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.255945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.257442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.257929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.258494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.259375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:47.259889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.261653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.270972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 406/807] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:47.272206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.293142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.300148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1d4c2-dcf1-4654-93a4-e21f52405a9b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.337990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.338660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.338938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.339216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/cb474b7f08db4ff9b9fdf8eb8ca658af/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.341174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=cb474b7f08db4ff9b9fdf8eb8ca658af, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.342601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.358149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 402/808] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:47 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/cb474b7f08db4ff9b9fdf8eb8ca658af/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:47.359272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.384493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072046-1e10-48b7-9268-49dd16263083 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.424282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.425160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.425582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.425994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.426825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:47.427369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.429106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.447458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 407/809] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:47.448244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.488601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.511367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734df48-6a91-4ca1-9c20-4f9aee0e7106 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.555146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.555990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.556275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.556547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.557196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.558775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.568638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb8977-1e11-424b-b32d-b599ccbe0138 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:47.569262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:47.570283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 06:16:47.571142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:47.574492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 403/810] 10.222.0.25 () {56 vars in 1129 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:47.575876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.605671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.614814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.615460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.616331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.616752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.634706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.637169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:47.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:47.658340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:47.659279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:47.659279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:47.659279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:47.662004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 408/811] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:47.663184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.714388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75076de-e636-4924-984e-c536c4e017c1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.763595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.764663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.764947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.765246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.771761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.776836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.778481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.792040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1b3ea-f830-46ad-a6a3-74371c670640 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:47.796352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:47.797428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:47.798594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:47.800776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 404/812] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:47.801769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.831513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.832163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.832435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.832705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.833323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:47.833623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.835851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.843868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 409/813] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:47.843868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.893170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:47.915478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186d03-5062-439e-854c-f68675a39846 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.951309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d4e4f-2b3e-454c-8e94-d56a7bef4abd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:47.952900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.954482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.954482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.954482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.954929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:47.955221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.956611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c532da8-a7a5-476d-9901-1bf7a2c4c173 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:47.969660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 405/814] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:47.970871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed76cd4-24b2-433b-bb22-560fa49f6f22 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.971153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed76cd4-24b2-433b-bb22-560fa49f6f22 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.971424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed76cd4-24b2-433b-bb22-560fa49f6f22 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.977592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 406/815] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:47.977592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed76cd4-24b2-433b-bb22-560fa49f6f22 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:47.994367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:47.995037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:47.995321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:47.995648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:47.996321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:47.996628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:47.998055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80c288-8f7d-427a-b37c-c0e72215b765 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.017283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 410/816] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:48.017283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.017283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.017283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.033518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:48.034700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:48.036683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.085133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed76cd4-24b2-433b-bb22-560fa49f6f22 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.111739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 411/817] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:48.112588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.135790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.136179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.136486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.136814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.137528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:48.137826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.139236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.149049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 407/818] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Sep 9 06:16:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:48.149049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.153522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.241991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b05d7-0cb9-4012-836a-62bfddd9214c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.281760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.282707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.283135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.283573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.292481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.294182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.296397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:48.309579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:48.310781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:16:48.312118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:48.315054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 412/819] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:16:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:48.316070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.381688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4752d-3ad6-45a2-9d02-9a46a7d1917f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.424624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.425293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.425821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.426099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/e0acbc7456fe40a1b0d70c0a35976fae/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.445919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f19dd-7f40-42e8-8c57-76085df9f9b3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.448171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=e0acbc7456fe40a1b0d70c0a35976fae, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.449596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.471593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 408/820] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:16:48 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/e0acbc7456fe40a1b0d70c0a35976fae/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:16:48.471593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.496483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.497204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.497590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.497930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.498764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.500603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.535688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 413/821] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:16:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:48.536846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.539375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.595248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab0fa-8490-4702-bbbd-61b1505d5ba6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.658330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.664207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.664853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.665131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.665398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.666153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:48.666456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.667986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.702765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 409/822] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:16:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:48.708795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.736178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cd955-1ec2-4fe7-b94b-8833b199b0e5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.789505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.790834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.791117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.791385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] PATH_INFO: `/v3/services/network` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.792295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.793959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:48.806857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:48.807991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:48.809427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 414/823] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:48 2022] GET /identity/v3/services/network => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:48.815109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:48.834788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5106c17-a956-44b5-bf27-2c6adae2e4ea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.846109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.878765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:48.879475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:48.879887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:48.880228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:48.960740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90065658-fe22-44f9-848f-97d30d1e1228 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:48.980278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:48.982227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:48.983946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.001727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:49.002504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.002879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.003267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.004138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] RBAC: Adding query filter params (name=network) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:49.004547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:49.006202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.014289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 415/824] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:16:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:49.018775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:49.162436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 410/825] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 06:16:49.164438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e5b5b-1aca-4960-96d2-aa2582644f88 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:49.168152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:49.190614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:49.209950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:49.210419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.210696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.211033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.211613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] RBAC: Adding query filter params (type=network) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:49.211916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:49.213281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.224708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 416/826] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:16:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:49.232145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:49.325277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:49.392303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faffa9-6181-4886-a188-5eb9bded1203 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:49.393863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673ac67-5f3d-4778-b2c7-cbe889a1553e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:49.455992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:49.456622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.456876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.457149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.457832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:49.459199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.465215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:49.465794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.466068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.466331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.487473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 417/827] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:16:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:49.492668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:49.494417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d6701230-1dec-4766-bb09-85a4003fd808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:49.516783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:49.517809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:49.518742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:49.518742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:49.518742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:49.520933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 411/828] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:49.527228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:49.677239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:49.686666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0358757-effe-4887-bc46-1eb280515b3f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:49.698977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a62a5c-0cdd-4db4-b0b0-9969a56b014e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.699309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a62a5c-0cdd-4db4-b0b0-9969a56b014e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.699529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a62a5c-0cdd-4db4-b0b0-9969a56b014e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.701111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 412/829] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:49.707103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.707736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.708214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.733854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:49.734392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.734671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.734943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.735622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:49.735929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:49.737306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.747741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:49.749289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:49.750402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 418/830] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:49.757999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:49.867580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:49.890708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 413/831] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:49.926129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278840-a562-49bc-9005-f2fef2a809d8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:49.963108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:49.964084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:49.964545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:49.965012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:49.972216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:49.973854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:49.986130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:49.987046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:49.987990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:49.990101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 419/832] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:49.996946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.105252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03296a-1566-4176-8b8c-0650db29ebc8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:50.144574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12436910-d9cb-4542-9300-1217a442341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:50.145435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12436910-d9cb-4542-9300-1217a442341a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.145863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12436910-d9cb-4542-9300-1217a442341a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.146282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12436910-d9cb-4542-9300-1217a442341a None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.147191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12436910-d9cb-4542-9300-1217a442341a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:50.147687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12436910-d9cb-4542-9300-1217a442341a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:50.149325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12436910-d9cb-4542-9300-1217a442341a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:50.161583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 414/833] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:50.168197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.190759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:50.230941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12436910-d9cb-4542-9300-1217a442341a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.283007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a403-abf2-4569-a4f3-3c4d0ffb6773 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:50.286087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:50.319334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:50.320002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.320276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.320552 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.321284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=dc56896d10094aed91db84ea2bf9dfb2) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:50.321593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:50.323122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:50.335599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 420/834] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:50 2022] GET /identity/v3/role_assignments?user.id=dc56896d10094aed91db84ea2bf9dfb2&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:50.349912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12436910-d9cb-4542-9300-1217a442341a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:50.388031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:50.388683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.388959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.389229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.401420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:50.407202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:50.408255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:50.409105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 415/835] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:50.417362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.516751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:50.536109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51b305-ae6c-4037-a4c0-1bb0b245333f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:50.581126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:50.581739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.582017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.582287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.582949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:50.583248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:50.584693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:50.593760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 421/836] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:50.599672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.711632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39e96c-1378-442e-ba49-e2c00252f058 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:50.722416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:50.752376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:50.752999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.753281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.753556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.764772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:50.766219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:50.781250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:50.782238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:50.783153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:50.783153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:50.783153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:50.783153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 416/837] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:50.790469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.900517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4fb7-425a-479c-9d02-3ae3bfc8bd8a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:50.939419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:50.940435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.940948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.941442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.942316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:50.942822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:50.944475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72366016-41cf-4994-99eb-cb47e791c1b5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:50.957825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 422/838] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:50.963784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:50.992312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c2dbb-a6cc-4335-915a-383803957680 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:50.992623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c2dbb-a6cc-4335-915a-383803957680 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:50.992863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c2dbb-a6cc-4335-915a-383803957680 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:50.995168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 423/839] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:51.001107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.001393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.001808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.014669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.022061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:51.023435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:51.093454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d8510-782b-4862-9416-064e75d87b94 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.099780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 424/840] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:51.132322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.133008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.133293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.133570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.142657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:51.144226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-746998dd-1fb5-402e-9117-0076dae6e8e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:51.157576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:51.158660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:51.159741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 417/841] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:16:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:51.165541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.268325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6a037-ae73-4928-be09-f1409f6294bf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.277223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf054-0fef-4a03-875f-1eb225328217 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.277582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf054-0fef-4a03-875f-1eb225328217 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.277924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf054-0fef-4a03-875f-1eb225328217 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.279446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 418/842] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:51.285658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.285938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.286350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.310413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:51.310930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.311636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:51.312007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.312475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.312967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.314005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:51.314608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:51.316253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:51.318318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.334854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.340281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 425/843] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:51.346763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.394926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 419/844] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:51.397423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.460924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4b43d-6ac3-4d87-b268-f3ed685bdfa9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.507437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.508220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.508598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.509001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.533096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8003c-f8c8-46c1-a24a-585de2087596 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.574952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.575905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.576407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.576817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.588341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.590961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:51.597272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:51.598189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:16:51.599031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:51.599697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:51.599769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:51.602627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 420/845] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:51.609229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.627340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.709921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 426/846] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:16:51.711187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.735950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44306e-ab9e-45f3-8533-9aeec42453af None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.775138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.775837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.776113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.776391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.777112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:51.777427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:51.778798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:51.787218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 421/847] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:51.788363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.813061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.823429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ecd84-113c-4b93-af51-525d8794a3db None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.861829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.862833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.862833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.863011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.863452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:51.864850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:51.868025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:51.874492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:51.875524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:51.876425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 427/848] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:51.877248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.901885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6078e1-454b-473e-895c-15ed96d49537 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:51.936859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:51.937495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:51.937771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:51.938045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:51.950022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:51.951449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:51.968910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:51.969878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:51.970808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:51.970808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:51.970808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:51.971799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 422/849] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:51.972513 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:51.985980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f51bc5-9fb1-498e-9557-61aef6ac9397 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.025082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.026085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.026427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.026702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.040809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.043077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:52.060574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:52.061640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:52.067570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:52.067570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:52.067570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:52.067570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 428/850] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:52.067570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.077939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02c925-4c5d-4327-9927-485e24afac6e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.082306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.106724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.118122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.118665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.118940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.119533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.120217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.120512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.121873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.126442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 423/851] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:52.127055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.185100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f0cb-224b-45d2-9f59-ea2262fdfe12 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.220229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.220945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.221283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.221625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.222327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.222697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.224084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.237067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 429/852] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:52.237883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.244422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280560c-d33a-4c7a-8ba0-63f7fa32d9bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.279673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.280286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.280563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.280830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.281467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.281766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.283340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.300180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 424/853] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:52.300180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.313861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.350318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d98a4-aad1-49cb-a56d-f8d9f68d52b1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.351654 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.388027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.388546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.388824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.389094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.389721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.390027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.391387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.396292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 430/854] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:52.397252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.420093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc813605-fc20-4ae3-9f7c-935bceafa306 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.477593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.478248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.478529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.479168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.486688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.488348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:52.505298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:52.506572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:52.507937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:52.510001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 425/855] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:52.517255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.553783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48198901-6504-434c-9ff0-c6db16692ade None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.598754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.609305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.610155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.610531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.610903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.620869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.622524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:52.640100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:52.641288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:52.642428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:52.649650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 431/856] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:52.650731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.660029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243e93-3d5f-4979-9259-2f7c2d5a0436 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.671249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.696825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.697443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.697721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.697989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.698684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.698985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.700552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.715380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 426/857] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:52.721987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.780249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce363e-3b5f-47f2-8eee-7158ea80554d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.818254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.819292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.819523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.819804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.820622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.820927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.822351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c4dee-a4d7-457e-9055-fdd34da260e0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.829524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826a60a-cfe8-4af7-894d-37275bc17cde None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:52.842300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 432/858] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:52.843485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a7c09-a1c0-4b3d-b611-093986d243ab None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.843808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a7c09-a1c0-4b3d-b611-093986d243ab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.844083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a7c09-a1c0-4b3d-b611-093986d243ab None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.845571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 433/859] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:52.847377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a7c09-a1c0-4b3d-b611-093986d243ab None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:52.868364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:52.869009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.869249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.869520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.870247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=cb474b7f08db4ff9b9fdf8eb8ca658af) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:52.870554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:52.872019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde74a9-d05d-4e16-bfbe-2c359a39780b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:52.887591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 427/860] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/role_assignments?user.id=cb474b7f08db4ff9b9fdf8eb8ca658af&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:52.887591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:52.887591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:52.887591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:52.906770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:52.908115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:52.919995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.947983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:52.997989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 428/861] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:53.013506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a7c09-a1c0-4b3d-b611-093986d243ab None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.055446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.056470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.057111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.057593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.132701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.134529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.192209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:53.239410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 434/862] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 06:16:53.244920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.268275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.352373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90496bdd-9efd-452c-96f4-b52e7d7a3fcd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.385102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbbd92-d2aa-428d-96d6-29e68f9c3692 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.390626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.391292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.391650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.391924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.406794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.409222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:53.425633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:53.426643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:53.427592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:53.427592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:53.427592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:53.431429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 429/863] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:53.432281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.432484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.432932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.433210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.433482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.434133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.435690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:53.446284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:53.448335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:16:53.449553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:53.451992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 435/864] 10.222.0.25 () {56 vars in 1132 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:53.453176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.457372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:53.476528 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:53.543004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f992993-e2ee-47cb-b9ef-f26266016af0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.578771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.579390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.579713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.579989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.580642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:53.580950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.582312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.589624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96673f64-944f-45d6-99e5-a2d491d265e9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.600175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 430/865] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:53.601111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.630873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.633111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.633978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.634698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.635573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:53.636127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.637891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.647955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 436/866] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:53.649375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.682038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:53.707310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b7634-963d-4e18-8bf1-fec9a55a54df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.745111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.746429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.746429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.746429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.753527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.754988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.756903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:53.768301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:53.769280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:53.770820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:53.774768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 431/867] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:53.775801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:53.791242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc2fcd-8cd7-474e-93a0-1813763e1063 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.834966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.836072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.836559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.837034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.837886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:53.838618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.840897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.852179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 437/868] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:53.893439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281c773-c4c2-4fb8-a89e-900d634211b8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:53.934192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:53.934839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:53.935179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:53.935459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:53.936259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:53.936570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:53.938070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b569f71-e32d-4c83-bffb-b24b82bf1ca9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:53.941382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:53.953737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 432/869] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:53.960207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:54.077267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b09ab0-0416-496e-8f69-9c6f58b7691a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:54.113978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:54.114952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:54.115471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:54.115975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:54.116917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=e0acbc7456fe40a1b0d70c0a35976fae) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:54.117438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:54.119093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba6932-1783-4de8-a79b-9c6c6f337de8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:54.125876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 438/870] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:53 2022] GET /identity/v3/role_assignments?user.id=e0acbc7456fe40a1b0d70c0a35976fae&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:54.704359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be7570-092a-4f5b-bf62-c7444079c1ae None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:54.704905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be7570-092a-4f5b-bf62-c7444079c1ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:54.704905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be7570-092a-4f5b-bf62-c7444079c1ae None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:54.706450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 433/871] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:54.712021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41270fd4-d6b7-46e0-abef-bdf3ae600edf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:54.712441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41270fd4-d6b7-46e0-abef-bdf3ae600edf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:54.712539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41270fd4-d6b7-46e0-abef-bdf3ae600edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:54.726722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:54.732497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-41270fd4-d6b7-46e0-abef-bdf3ae600edf None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:54.734333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-41270fd4-d6b7-46e0-abef-bdf3ae600edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:54.806683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 439/872] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:55.091497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be7570-092a-4f5b-bf62-c7444079c1ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:55.202386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be7570-092a-4f5b-bf62-c7444079c1ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:55.225203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68a93-9d1b-41bc-9078-c17db30e5b79 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:55.225480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68a93-9d1b-41bc-9078-c17db30e5b79 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:55.225759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb68a93-9d1b-41bc-9078-c17db30e5b79 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:55.227259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 440/873] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:55.232797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:55.233083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:55.233363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:55.241963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:55.242832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:55.243289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:55.243771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:55.244660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:55.246248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:55.259573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:55.259573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:55.268159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 434/874] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:16:55 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:55.351796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 441/875] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:55.626709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:55.684994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:55.737771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e96ce-eb19-4bb9-b477-6e0746e62fb8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:55.774375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:55.775012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:55.775292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:55.775583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:55.776185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:55.777616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:55.786801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:55.787885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:55.788757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 435/876] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:16:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:55.796650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:55.865800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:55.906064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf421ca-bb6b-44d2-b103-fabfa31156b0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:55.941988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:55.942609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:55.942887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:55.943164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:55.943957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:55.944259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:55.945631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbecb14-28e3-438d-87ee-f3cad5ac307b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:55.955300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 442/877] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:16:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:55.959734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:56.073806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec8e60-ba30-416a-ab88-f61954429a21 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:56.095986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:56.111458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:56.112101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:56.112384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:56.112658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:56.113298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:56.113597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:56.114960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86a5c3-9580-40c3-9311-ce7d6166531b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:56.120655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 436/878] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:16:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:56.147534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b429-79a2-4785-80b3-c290e011ed76 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:56.148234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b429-79a2-4785-80b3-c290e011ed76 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:56.148744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b429-79a2-4785-80b3-c290e011ed76 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:56.150755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 443/879] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:56.156964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:56.157591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:56.158083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:56.179294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:56.181060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:56.263605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 437/880] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:56.533273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3652-8b38-4c75-9257-e2e80bffa921 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:56.533897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3652-8b38-4c75-9257-e2e80bffa921 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:56.534388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3652-8b38-4c75-9257-e2e80bffa921 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:56.536034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 444/881] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:56.537300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:56.545135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:56.548089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:56.548577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:56.572619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:56.573823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:56.653065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 445/882] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:16:56.671146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576951a-afc5-4f17-bad6-b92eaab131ee None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:56.686644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:56.709423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:56.710278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:56.710716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:56.711145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:56.711986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:56.713566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:56.730283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 438/883] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:16:56 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:16:56.950243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.011087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:57.064198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cba26c-2648-4b44-af71-22688708ce58 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.108116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:57.108749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.109025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.109294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.120173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:16:57.124988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:57.126000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:57.126926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 446/884] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Sep 9 06:16:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:57.134524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.243353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc735411-f00f-4b76-b978-eb3416a4d730 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.279428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:57.280484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.281014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.281485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.282390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:57.282901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:57.284483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:57.293893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 439/885] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:16:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:57.299958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.411287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38496e4f-42fc-4655-9f4a-7652c61ac975 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.425806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:57.449732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:57.450368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.450645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.450921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.462908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:57.464399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-de6b12d0-51ec-410b-a001-b933a03fa879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:57.480529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:57.481680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:57.482593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:57.482593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:57.482593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:57.482593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 447/886] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:57.488978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.493892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71a9d2-746e-44c1-aa9b-cabd910be17b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.494139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71a9d2-746e-44c1-aa9b-cabd910be17b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.494365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71a9d2-746e-44c1-aa9b-cabd910be17b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.495987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 448/887] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:16:57.501095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.501378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.501746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.521139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:57.523227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:57.527834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:57.596866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 449/888] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:57.600268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69291a5c-c89d-4b17-aa15-464d0cab39ee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.636208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:57.636821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.637103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.637374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.638016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:57.638318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:57.639703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:57.652241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 440/889] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:57.657956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.755233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:57.762991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cdd70-e744-4857-a3d7-ff1120a88a1d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.798091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:57.798725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.799001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.799273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.806823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:57.808353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:16:57.820558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:57.821519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:57.822484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 450/890] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:16:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:57.827607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.863110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:57.942200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:57.954077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56c090-db78-44bb-a6a5-647845e2a5d5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.993444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ac82-0fbc-4175-bf9b-2d20c573ff46 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:57.997307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:57.997929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:57.998206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:57.998475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:57.999211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:57.999514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.000985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.026166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 441/891] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Sep 9 06:16:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:58.027035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.033839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.034800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.035294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.035900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.036686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.038679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:58.049323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:58.050415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:16:58.051292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:16:58.059501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 451/892] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:16:58.060771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.083703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.133276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112e4a9-a5d5-4017-b280-534bb07bd6d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.168336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.168969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.169203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.169475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.176911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bf62e-f61a-42a4-9f1d-aa0f8fa1a26e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.195068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.219378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.220373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.220869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.221357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.222259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:58.222817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.224402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.229907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 452/893] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:58.237050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.242637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.244112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.327892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.348844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3547-31db-4243-8923-33fc80f267a1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.352486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 442/894] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:16:58.353265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.392299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.393549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.394034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.394526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.395394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:58.395931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.397951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.404384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 453/895] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:58.410335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.416208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.464169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89fde-557b-4937-9e67-e10a7cfcf11d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.499921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.500552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.500833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.501102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.511788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.513221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.522050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa820-24e3-46db-b73b-cb8dfdbd117b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-64047478-c8cf-4d2a-baa6-155fa074caf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:16:58.528817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:58.529785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:16:58.530820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:16:58.530820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:16:58.530820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:58.530820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 443/896] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:58.531650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83addb-12e2-4b06-91a6-fda8cfd8bea0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.531650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83addb-12e2-4b06-91a6-fda8cfd8bea0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.531650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83addb-12e2-4b06-91a6-fda8cfd8bea0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.533008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 444/897] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:58.536554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83addb-12e2-4b06-91a6-fda8cfd8bea0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.566986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.567624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.567910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.568690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.569404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] RBAC: Adding query filter params (interface=public, service_id=dd8350be0f68453f8aebbf0afca4f389, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:58.569712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.571082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c9b77-c758-43d5-87ce-7573453d37a6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.577302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 454/898] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/endpoints?service_id=dd8350be0f68453f8aebbf0afca4f389&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:58.578817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.579036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.579340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.598243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:58.599962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:58.603808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.658794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83addb-12e2-4b06-91a6-fda8cfd8bea0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.666508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.673221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 455/899] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:58.676727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.696731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.697291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.697571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.697890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.698553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:58.698895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.700498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.716238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 445/900] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:58.721308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.808265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce02a4-9db6-4ce9-a31c-90529fa16bf2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.828487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6d4e3-be67-4d2e-971d-f7049c0c6093 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:58.848277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.849234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.849880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.850372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.851178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.853026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.862789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 456/901] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:16:58.879616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:58.879616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:58.879616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:58.879616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:58.883792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:58.885418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:16:58.900647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:58.901656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:16:58.902566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:58.906797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 446/902] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:58.908307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.916732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:58.945886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:58.978204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:59.033885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3e68d-8fb6-41dd-9863-3a7ab7a83106 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:59.072974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da44edd-a94b-4819-9b1a-18054ab37d86 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:59.073200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:59.073477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.073759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.074035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.074731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:59.075044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:59.076577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7635322-1975-4157-86ef-22c8fad4f845 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:59.089229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 457/903] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:59.090633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8bc5b-7636-46fc-abc4-f662ebc1d03b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.090979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8bc5b-7636-46fc-abc4-f662ebc1d03b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.091262 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8bc5b-7636-46fc-abc4-f662ebc1d03b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.092787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 458/904] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:16:59.095528 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8bc5b-7636-46fc-abc4-f662ebc1d03b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:59.114653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b8fda-890e-4c3f-90c8-336a2c4f739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:59.116506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8fda-890e-4c3f-90c8-336a2c4f739d None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.116948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8fda-890e-4c3f-90c8-336a2c4f739d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.117358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8fda-890e-4c3f-90c8-336a2c4f739d None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.118169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b8fda-890e-4c3f-90c8-336a2c4f739d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:59.120500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b8fda-890e-4c3f-90c8-336a2c4f739d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:59.139400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 447/905] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:16:58 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:16:59.145863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.146519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.147325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.171235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:59.175070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:16:59.177373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:16:59.237027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8bc5b-7636-46fc-abc4-f662ebc1d03b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:59.275582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 448/906] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:16:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:16:59.285962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:59.286605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.286926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.287250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.288172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] RBAC: Adding query filter params (role.id=e1edcc38ef8d4a7982d84cff4e923c8d, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=e0acbc7456fe40a1b0d70c0a35976fae) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:59.288489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:59.290124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:59.297354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 459/907] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:16:59 2022] GET /identity/v3/role_assignments?user.id=e0acbc7456fe40a1b0d70c0a35976fae&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=e1edcc38ef8d4a7982d84cff4e923c8d => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:16:59.545169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:59.562354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:59.657794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abaccd-cb37-4852-9278-8422e49b7e12 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:59.694755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:59.695454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.695767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.696044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.696639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:59.698105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:16:59.707776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:16:59.709264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:16:59.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 449/908] 10.222.0.25 () {56 vars in 1132 bytes} [Fri Sep 9 06:16:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:16:59.717985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:59.826766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea101ace-8357-4f7f-b3f5-0f30a1e8e601 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:16:59.852715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:59.864459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:16:59.865819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:16:59.866127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:16:59.866404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:16:59.867051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:16:59.867360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:16:59.868799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:16:59.874327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 460/909] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:16:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:16:59.880752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:16:59.970336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:16:59.991900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8087-3b33-49eb-b080-f4cf2b882ec8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:00.027764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f3478-460a-41ef-8489-029826cade50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:00.028380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f3478-460a-41ef-8489-029826cade50 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:00.028663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f3478-460a-41ef-8489-029826cade50 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:00.028938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f3478-460a-41ef-8489-029826cade50 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:00.029577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f3478-460a-41ef-8489-029826cade50 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:00.029882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f3478-460a-41ef-8489-029826cade50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:00.031254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f3478-460a-41ef-8489-029826cade50 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:00.036571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 450/910] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:16:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:00.044626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:00.153701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f085a5-d607-4b8d-8066-136400b4a4a7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:00.190781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:00.191410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:00.192124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:00.192124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:00.193938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] RBAC: Adding query filter params (interface=public, service_id=d28fc5631a574a59baff65709c9fe7c3, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:00.193938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:00.194892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c2c1-abd2-4425-9cd1-5f42d6481ee2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:00.201311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 461/911] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:17:00 2022] GET /identity/v3/endpoints?service_id=d28fc5631a574a59baff65709c9fe7c3&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:00.208631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f3478-460a-41ef-8489-029826cade50 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:00.316484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f3478-460a-41ef-8489-029826cade50 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:00.352581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c7bda-44ab-41f0-b0a9-9e4bf388b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:00.353191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c7bda-44ab-41f0-b0a9-9e4bf388b69c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:00.353469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c7bda-44ab-41f0-b0a9-9e4bf388b69c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:00.353740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c7bda-44ab-41f0-b0a9-9e4bf388b69c None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:00.354380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c7bda-44ab-41f0-b0a9-9e4bf388b69c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:00.355738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c7bda-44ab-41f0-b0a9-9e4bf388b69c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:00.362073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 451/912] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:17:00 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:17:01.220931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472066-66c9-471e-81de-7b2e1b112f7a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.221479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472066-66c9-471e-81de-7b2e1b112f7a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.221479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472066-66c9-471e-81de-7b2e1b112f7a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.223036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 462/913] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:01.228908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12570678-0753-4a80-a5cc-eeee5a2e88da None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.229762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12570678-0753-4a80-a5cc-eeee5a2e88da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.230245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12570678-0753-4a80-a5cc-eeee5a2e88da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.250184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-12570678-0753-4a80-a5cc-eeee5a2e88da None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:01.251825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-12570678-0753-4a80-a5cc-eeee5a2e88da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:01.254400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:01.332222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 452/914] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:01.380049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64327f-42cf-408b-b6cd-9e5ab9a127da None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.380880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64327f-42cf-408b-b6cd-9e5ab9a127da None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.381632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64327f-42cf-408b-b6cd-9e5ab9a127da None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.383416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 463/915] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:01.388327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5326be-abf6-4cf2-b191-963c054feb2f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.388822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5326be-abf6-4cf2-b191-963c054feb2f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.389303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5326be-abf6-4cf2-b191-963c054feb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.415873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7d5326be-abf6-4cf2-b191-963c054feb2f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:01.418373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7d5326be-abf6-4cf2-b191-963c054feb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:01.504496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 453/916] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:01.551253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bdbf4-102b-4705-b42f-b72e6cb1817e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.551555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bdbf4-102b-4705-b42f-b72e6cb1817e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.551856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bdbf4-102b-4705-b42f-b72e6cb1817e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.553603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 464/917] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:01.558912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.559209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.559484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.584241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:01.585432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:01.639685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bdbf4-102b-4705-b42f-b72e6cb1817e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:01.662369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:01.671634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 454/918] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:01.703565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:01.752430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bdbf4-102b-4705-b42f-b72e6cb1817e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:01.777255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:01.787314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:01.787987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.788264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.788535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.789119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:01.790476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:01.799365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:01.800411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:01.801249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 465/919] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:17:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:01.808399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:01.901095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7339621-1b48-4671-bc94-193aaa7b617c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:01.926352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9aaf-a86d-4551-8b44-305acb78e48f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:01.940966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:01.941923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.942343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.942750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.943780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:01.944536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:01.946582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:01.957659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:01.959351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:01.960244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:01.962897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 455/920] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:17:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:01.966602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:01.967365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:01.967487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:01.968020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:01.968506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:01.969275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:01.969771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:01.971367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:01.976676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 466/921] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:17:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:01.977644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.034630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.075690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4caf06-fe50-4bef-b6e0-66179b38a253 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.084571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa57c-713a-40bd-a440-7dbe390a7c81 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.112430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.113170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.113446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.113720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.119244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.119887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.120198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.120437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.121062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:02.121363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.122724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.124950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:02.127934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 467/922] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:02.130218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:02.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:02.136795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 456/923] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Sep 9 06:17:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:02.137506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.137594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.185739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.242746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd901e2c-c8e8-480c-a985-061b1892dc49 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.246268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f65c70-2ace-4173-9bbf-82548205919c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.266726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.280320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.280855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.280985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.281201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.281458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.281568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.281784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.282059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.282155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:02.282394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.282675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:02.282977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.283794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.284337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.294412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 457/924] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:02.294573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 468/925] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:02.294681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.295360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.396771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3fa38-cb3c-43a3-97e5-5069d9883957 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.408942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d96a20-7a99-463c-a09a-da93980198b4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.419634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.431471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.432117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.432397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.432670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.433346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:02.433638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.435029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.442926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 458/926] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:02.443804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.456854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.457475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.457752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.458021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.458778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] RBAC: Adding query filter params (interface=public, service_id=77279ffe447148daa845c04c4e0ed7e6, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:02.459090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.460477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.470159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 469/927] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/endpoints?service_id=77279ffe447148daa845c04c4e0ed7e6&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:02.470159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.510281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.570623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84452-cb42-4edb-98a1-002aafc71d08 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.589627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e614b-1436-4cfd-9256-1341a1ce9313 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.606755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.607366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.607675 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.607949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.608622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.609977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.626378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.627157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.627748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.628024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.632587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 459/928] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:17:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:02.633401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.641489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.643038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a2c642dc-24f0-49d6-a14f-7e053e6a807d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:02.658653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:02.659912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:02.661040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:02.661040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:02.661040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:02.663746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 470/929] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:02.666113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972de40-d157-42af-ab8f-074adc629b4e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.666402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972de40-d157-42af-ab8f-074adc629b4e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.666841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972de40-d157-42af-ab8f-074adc629b4e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.668669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 471/930] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:02.672488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972de40-d157-42af-ab8f-074adc629b4e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.707188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.762190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:02.767611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92d445-612e-4c40-a21e-9942f9962911 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.801564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972de40-d157-42af-ab8f-074adc629b4e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:02.815884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344d44f-ef08-4851-99ff-d7e0402aa6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.816505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344d44f-ef08-4851-99ff-d7e0402aa6ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.816785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344d44f-ef08-4851-99ff-d7e0402aa6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.817057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344d44f-ef08-4851-99ff-d7e0402aa6ad None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.817676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344d44f-ef08-4851-99ff-d7e0402aa6ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.819857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344d44f-ef08-4851-99ff-d7e0402aa6ad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.827065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 460/931] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:17:02.831527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.831527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.831527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.848895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:02.850575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:02.850934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:02.852760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:02.852760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:02.852760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:02.853119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:02.853426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:02.854829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:02.869339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 472/932] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:02.875263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:02.943432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 461/933] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:02.988783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:03.007878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa5966-5120-468b-ae23-b8e3225fa627 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.044675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.045319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.045598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.045873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.053978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.055367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:17:03.069349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:03.070426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:17:03.071378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:03.075341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 473/934] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:17:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:03.080685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.182715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:03.190968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96bb5e-cb90-434c-9142-4538f0f3412f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.217370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.234819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.235441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.235748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.236021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.236768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:03.237076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.238512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.262156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 462/935] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:03.264111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.354833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2fee-61dd-4940-b12b-d21403de0939 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.372745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb4c0e-2506-4d29-b7eb-a84f84d72441 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.397757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.398783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.399267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.399770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.400573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.402635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.410573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.411215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.411494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.411848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:03.414533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:03.415700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 06:17:03.416804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:03.419572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 474/936] 10.222.0.25 () {56 vars in 1132 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:03.425121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.432227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.433975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.439027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:03.457153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:03.458172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:03.459166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:03.459166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:03.459166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:03.461216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 463/937] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:03.466418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.470273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:03.552131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20efa-87cd-433e-9650-2ed3f5bb9917 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.596799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.598031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.598432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.599083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.600378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:03.600798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.602383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.604930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a299b-67d1-4658-a2a5-fb80f0798d2a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.614348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 475/938] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:03.616189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.649899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.650921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.651377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.652130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.653215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:03.653666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.655664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.671489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 464/939] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:03.677564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.711660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:03.746308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814de4-eaa9-4e66-ab2b-63b3e18bb7ea None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.765025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:03.794827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.796134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.796626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.797193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.798121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:03.798627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.800638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.811532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07c2c7-73d0-412b-a67b-3e9bc7aa1f6b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.812380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 476/940] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:03.814303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.848694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.849911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.850196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.850470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.857762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.859137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:17:03.871375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:03.872322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 465/941] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:03.878148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:03.935483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466ef5e-dace-44e9-be2d-0b6d514ba571 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:03.974634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:03.975587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:03.975872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:03.976221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:03.976807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:03.978160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:03.985148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:04.007574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 477/942] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:17:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:04.032609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae93a6-2aaf-4261-a95c-fb513c29746d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:04.061446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:04.084014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:04.084801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:04.085353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:04.085641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:04.086347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:04.086652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:04.088240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ea50a-951b-4bf7-a150-814ea15f172a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:04.103317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 466/943] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:17:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:04.111450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:04.232968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5341c-4e61-4ab3-a903-8a9493b58b2d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:04.268435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c5be5-cadd-4433-b344-925df5f3eef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:04.268993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c5be5-cadd-4433-b344-925df5f3eef5 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:04.269276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c5be5-cadd-4433-b344-925df5f3eef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:04.269595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c5be5-cadd-4433-b344-925df5f3eef5 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/e0acbc7456fe40a1b0d70c0a35976fae/roles/e1edcc38ef8d4a7982d84cff4e923c8d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:04.272573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c5be5-cadd-4433-b344-925df5f3eef5 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=e0acbc7456fe40a1b0d70c0a35976fae, role_id=e1edcc38ef8d4a7982d84cff4e923c8d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:04.274000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c5be5-cadd-4433-b344-925df5f3eef5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:04.292853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 478/944] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:17:04 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/e0acbc7456fe40a1b0d70c0a35976fae/roles/e1edcc38ef8d4a7982d84cff4e923c8d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:17:05.032957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f2d02-98ab-472c-82c9-f53d78b6d995 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.033481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f2d02-98ab-472c-82c9-f53d78b6d995 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.033481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f2d02-98ab-472c-82c9-f53d78b6d995 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.035073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 467/945] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:05.041623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.041623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.041623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.061947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:05.063481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:05.117161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:05.126101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e8e6-19db-4c38-bea1-33f9a7b169b1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.126420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e8e6-19db-4c38-bea1-33f9a7b169b1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.126698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279e8e6-19db-4c38-bea1-33f9a7b169b1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.128326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 468/946] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:05.133806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.134093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.134370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.139392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 479/947] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:05.153065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:05.154258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:05.231699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 469/948] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:05.414327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:05.504823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:05.517583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de90e7-07de-4318-9e16-c4cc8017ffa2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:05.557969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:05.558838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.559116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.559387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.560072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:05.561604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:05.566843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING keystone.server.flask.common [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 06:17:05.586890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.sql.core [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 06:17:05.586890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 06:17:05.586890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [parameters: {'id': '47d35467e1d942a0a5e2602ca3cce03e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 06:17:05.586890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47d35467e1d942a0a5e2602ca3cce03e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47d35467e1d942a0a5e2602ca3cce03e. Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 06:17:05.607921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 06:17:05.608871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application yield resource Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise self.value Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application session.commit() Sep 09 06:17:05.609882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.flush() Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application compat.raise_( Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 06:17:05.610809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 06:17:05.616802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 06:17:05.617622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [parameters: {'id': '47d35467e1d942a0a5e2602ca3cce03e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.618665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:05.619641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 47d35467e1d942a0a5e2602ca3cce03e. Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.620735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 480/949] 10.222.0.25 () {60 vars in 1140 bytes} [Fri Sep 9 06:17:05 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:05.623616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:05.653899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e0b99-4dfc-4868-b9c8-4423a0a471ce None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:05.669270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:05.671198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:05.700512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:05.701434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.701965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.702393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.703521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:05.705120 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:05.715249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:05.716464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:05.717317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:05.725269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 470/950] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:17:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:05.726331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:05.758112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3ffb-79a5-43a2-b4cf-70a1815539a4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:05.805378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:05.806134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.806511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.806849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.815750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:05.818621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:17:05.835745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:05.836689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:05.837733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:05.839047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9111ae-6aca-4145-b964-9f9cb506e5a8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:05.842100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 481/951] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:05.843942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:05.877259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:05.878142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:05.878601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:05.879149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:05.880271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:05.880784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:05.882566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:05.892978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 471/952] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:05.894124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:05.945550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:05.964468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:05.970465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c406f9-abd6-4ab5-9328-da2d90ff1560 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:06.011595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:06.011595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.011595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.011595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.012022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:06.012396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:06.014007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:06.022131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa834-1820-4746-a535-ed45cbf6f0a7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:06.035743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 482/953] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:06.067003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:06.068875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.069309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.069754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.070570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:06.071083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:06.072790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68734c-666a-4d96-8ad9-636a0e867e81 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:06.079393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 472/954] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:06.091587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:06.211393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81944f-8e53-4220-a055-0ed4b8522e46 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:06.249460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bd059-17c0-428a-90ef-215edfbd778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:06.250068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bd059-17c0-428a-90ef-215edfbd778a None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.250348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bd059-17c0-428a-90ef-215edfbd778a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.250619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bd059-17c0-428a-90ef-215edfbd778a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.251299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bd059-17c0-428a-90ef-215edfbd778a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:06.252771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bd059-17c0-428a-90ef-215edfbd778a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:06.276947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 483/955] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:17:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:17:06.574997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced57a-e804-47bd-8fe3-c679cffe3189 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.575318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced57a-e804-47bd-8fe3-c679cffe3189 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.575573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ced57a-e804-47bd-8fe3-c679cffe3189 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.577045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 473/956] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:06.582847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.583989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.584498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.602169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b616f1f-9a6f-4877-88af-68bffaf26d38 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.602482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b616f1f-9a6f-4877-88af-68bffaf26d38 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.602757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b616f1f-9a6f-4877-88af-68bffaf26d38 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.604358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 474/957] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:17:06.604854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:06.606597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:06.609759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:06.610061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:06.610323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:06.617777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:06.628991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:06.631161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:06.704563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 484/958] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:06.719652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 475/959] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:06.980283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:06.995102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.029343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:07.107190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4612d-417d-472c-985b-de967fa4e6e1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.110236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b83f3-0c5d-47d8-a166-22b00cec62b5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.149704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.150927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.151560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.152052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.152929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.153379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.154015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.154296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.154571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.155612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:07.166036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:07.167138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:07.168087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:07.168864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:07.170891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 485/960] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:07.171266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:07.172316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:07.173440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:07.182183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.182315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 476/961] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Sep 9 06:17:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:07.184474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.185645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:07.289386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:07.300334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddef68-0c25-42d1-b642-40c4c7d40801 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.312549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ae260-cc6b-48b2-b15d-45b5c68c85c3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.339240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.340244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.340753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.341278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.342235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:07.342732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.344419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.353525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.354144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.355269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.355269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.355458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:07.355661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.359668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.359931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 477/962] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:07.361480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.362557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 486/963] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:07.369547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.450099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:07.486167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c7835-f995-4285-bfb4-5ce7ef4459f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.495127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57306695-4c47-447c-bfb4-2d95680c005c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.527325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.528250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.528537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.528816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.538291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.539230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.539827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.540387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.541206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:07.541713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.542311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.543587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.543768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.550232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 487/964] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:07.564464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:07.565543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:07.566497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:07.566497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:07.566497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:07.574565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 478/965] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:07.576372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.687515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07164d2-19bd-4e46-a0fa-1cfd5dc34465 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.721148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:07.728022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.728651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.728929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.729198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.729849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:07.730150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.731556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.749382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 488/966] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:07.755017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:07.765583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:07.877159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7e533-8203-4ac2-a543-6c527786eab2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:07.915248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:07.916176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:07.916599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:07.917012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:07.925347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:07.927760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:17:07.940015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:07.941236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 06:17:07.942436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:07.948530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 479/967] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:07.951268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.056218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b0900-6292-495e-a9b2-e23a3da48f49 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:08.092749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:08.093366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.093643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.093914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.094642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:08.094945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:08.096507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a8e9e-39e8-49d8-8da2-71ecfd7691d5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:08.117842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 489/968] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Sep 9 06:17:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:08.124302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.197829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:08.239946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689385a-e56d-4bff-86f6-85c1d87c2517 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:08.276301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab0108-2fcd-4ff5-89b5-c6fd7bb49f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:08.277269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab0108-2fcd-4ff5-89b5-c6fd7bb49f7f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.277773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab0108-2fcd-4ff5-89b5-c6fd7bb49f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.278242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab0108-2fcd-4ff5-89b5-c6fd7bb49f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.318323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe835ab-4aed-456b-af4e-a989a2b4807a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.318782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe835ab-4aed-456b-af4e-a989a2b4807a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.319044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe835ab-4aed-456b-af4e-a989a2b4807a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.320613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 490/969] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:08.325888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.326168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.326442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.344706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:08.345884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:08.354584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab0108-2fcd-4ff5-89b5-c6fd7bb49f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:08.356494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab0108-2fcd-4ff5-89b5-c6fd7bb49f7f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:08.398539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:08.429847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 491/970] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:08.448100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:08.475723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 480/971] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:08.478144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.591351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93666e5c-cfd4-406a-9c90-6454cbb25f7f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:08.595485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d59cf6-030d-4f26-9805-85f5faca5477 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.596024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d59cf6-030d-4f26-9805-85f5faca5477 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.596531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d59cf6-030d-4f26-9805-85f5faca5477 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.598313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 481/972] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:17:08.603533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.603971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.604351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.622923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:08.624288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:08.629960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:08.631211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.631668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.632105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.644186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:08.646462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:08.665056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:08.666048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:08.666959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:08.666959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:08.666959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:08.670830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 492/973] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:08.672042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.707647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 482/974] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:08.747914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:08.754692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.785625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121679b-2d4d-48b5-ae42-170e024d635c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:08.824935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:08.825962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.826336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.826705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.827458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:08.827880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:08.829713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:08.844474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 493/975] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:08.850422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.875366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0dff3b-8b7f-49cc-a3b0-1bf82eb538be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:08.899966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:08.910150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:08.910888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:08.911214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:08.912160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:08.912941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:08.914492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:08.917180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING keystone.server.flask.common [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 06:17:08.930125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 06:17:08.930125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 06:17:08.930125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [parameters: {'user_id': 'ed31d5d13fc240aaabe984fdc7f7a3f3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 06:17:08.930125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Sep 09 06:17:08.937709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 06:17:08.938621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 06:17:08.939936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 06:17:08.941026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 06:17:08.942455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 06:17:08.946579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ed31d5d13fc240aaabe984fdc7f7a3f3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:08.947952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 06:17:08.948944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 483/976] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Sep 9 06:17:08 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:08.952484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:08.982327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:08.994062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e54e-12b1-4bd3-9295-60850de7fb14 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.032018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.032652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.032928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.033199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.040562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.041982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:17:09.053791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:09.054704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 494/977] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:09.056066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.082774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9f71b-3dbc-452e-8ef5-266a8426e6cc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.119221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.120009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.120288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.120556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.128703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.130858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:17:09.145928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:09.147066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 484/978] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:09.153327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.170712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645c58f-eb96-493e-90f9-925ab4f8961e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.171202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:09.206684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.207709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.208322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.208671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.209326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.210780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:09.219815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:09.220830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:09.222106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 495/979] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:09.222973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.246617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:09.261627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e67e0-086c-4c74-a531-4b08533efc7a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.296548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.297183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.297442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.297711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.298519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:09.298828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.300382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.319920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 485/980] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:09.321524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.340289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e034c9b-128a-4f0d-ae5d-2ba40bd97529 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.375908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.376519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.376793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.377055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.377751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:09.378052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.379464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.416342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 496/981] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:09.417160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.431849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:09.458748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2e97-208c-4e6f-b9e6-190cbf83865f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.509138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.509673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.509949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.510215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.510854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:09.511186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.512082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:09.513162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.524191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 486/982] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:09.529935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.555604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3402-e568-4d1e-a2c5-d550041cb449 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.603389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.604198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.604542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.604886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.605703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] RBAC: Adding query filter params (role.id=e1edcc38ef8d4a7982d84cff4e923c8d, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=e0acbc7456fe40a1b0d70c0a35976fae) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:09.606080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.607669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.620036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 497/983] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/role_assignments?user.id=e0acbc7456fe40a1b0d70c0a35976fae&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=e1edcc38ef8d4a7982d84cff4e923c8d => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:09.671592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c178c5-d0ba-4969-8c90-18df2febb8e6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.714534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.715500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.716031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.716492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.717384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:09.717909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.719599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815bd5e-7082-4121-8d08-48fc3705a50a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.726463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 487/984] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:09.733087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.848386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851613b1-8917-4ad0-bf03-71c7ad5a252f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.848649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851613b1-8917-4ad0-bf03-71c7ad5a252f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.849394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851613b1-8917-4ad0-bf03-71c7ad5a252f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.849526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a029e-177d-46da-a4af-38334b195e1e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:09.851269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 488/985] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:17:09.857244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.857530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.857810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.872138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:09.884138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:09.885685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:09.887537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:09.888183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:09.888455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:09.888728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:09.889434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] RBAC: Adding query filter params (interface=admin, service_id=77279ffe447148daa845c04c4e0ed7e6, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:09.889746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:09.891100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:09.896909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 498/986] 10.222.0.25 () {56 vars in 1249 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/endpoints?service_id=77279ffe447148daa845c04c4e0ed7e6&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:09.903194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:09.906390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:09.961320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 489/987] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:10.014315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941856d-3577-4003-b6f4-556035c87c6d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:10.049374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464c1b-c601-4dd1-a483-e579b8abc36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:10.049976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464c1b-c601-4dd1-a483-e579b8abc36f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:10.050254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464c1b-c601-4dd1-a483-e579b8abc36f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:10.050524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464c1b-c601-4dd1-a483-e579b8abc36f None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:10.051123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94464c1b-c601-4dd1-a483-e579b8abc36f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:10.052807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94464c1b-c601-4dd1-a483-e579b8abc36f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:10.067592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 499/988] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:17:09 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:17:10.230478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:10.332428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9fa8b-f96b-43a4-adb8-09eb8a3c9f35 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:10.367151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728694c-7454-477b-85bf-95ef82d2b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:10.367789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728694c-7454-477b-85bf-95ef82d2b4d3 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:10.368073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728694c-7454-477b-85bf-95ef82d2b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:10.368347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728694c-7454-477b-85bf-95ef82d2b4d3 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:10.369004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728694c-7454-477b-85bf-95ef82d2b4d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:10.370378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728694c-7454-477b-85bf-95ef82d2b4d3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:10.374716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:10.388810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 490/989] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:17:10 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:11.867685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137f9bc-d895-41ac-b1ea-77d9f74b1774 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:11.868276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137f9bc-d895-41ac-b1ea-77d9f74b1774 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:11.868276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137f9bc-d895-41ac-b1ea-77d9f74b1774 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:11.869538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 500/990] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:11.874833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a93f0-4149-4ce6-b775-036d453d322e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:11.875208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a93f0-4149-4ce6-b775-036d453d322e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:11.875381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a93f0-4149-4ce6-b775-036d453d322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:11.896824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bb4a93f0-4149-4ce6-b775-036d453d322e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:11.898024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb4a93f0-4149-4ce6-b775-036d453d322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:11.978160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 491/991] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:12.215120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142bab3-e22f-464b-9fae-eac0b0e5f5ea None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.215503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142bab3-e22f-464b-9fae-eac0b0e5f5ea None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.215719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142bab3-e22f-464b-9fae-eac0b0e5f5ea None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.217166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 501/992] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:12.222161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.222451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.222722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.241820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:12.242999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:12.247232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142bab3-e22f-464b-9fae-eac0b0e5f5ea None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:12.315960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 492/993] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:12.324100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:12.360769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142bab3-e22f-464b-9fae-eac0b0e5f5ea None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:12.395784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:12.396555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.396829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.397098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] PATH_INFO: `/v3/services/image` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.397674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:12.399156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:12.408044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:12.409062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:12.409928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 502/994] 10.222.0.25 () {56 vars in 1105 bytes} [Fri Sep 9 06:17:12 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:12.415352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:12.475946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:12.527117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a6732-d1d5-4117-9731-fcd73c292902 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:12.562348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:12.562973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.563253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.563524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.564178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:12.564480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:12.565841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e906dce-b9f1-477b-b4dc-3f06c7be29e7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:12.575352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 493/995] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:12.576964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:12.577272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b29dc-a543-48f5-8a46-e6c914401d29 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.577565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b29dc-a543-48f5-8a46-e6c914401d29 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.578098 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b29dc-a543-48f5-8a46-e6c914401d29 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.583939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 494/996] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:12.585115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.585394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.585671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.603925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:12.605119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:12.670370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 495/997] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:12.671193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:12.683490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bc94-21cf-43a2-bf6c-a2f3a3dc2d54 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:12.715877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:12.722564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:12.723102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:12.723218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.723436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.724289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.725016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] RBAC: Adding query filter params (type=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:12.725333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:12.727142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:12.731590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 503/998] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:12.782957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812f39c-7d84-4aa1-a377-82d41fa9e2f3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:12.822847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:12.823508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.823822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.824099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.824713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:12.826127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:12.836227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:12.837407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:12.840691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 496/999] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Sep 9 06:17:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:12.843428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:12.944208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:12.956506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd8adb-aec5-4868-8b59-3bbf48a81248 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:12.992184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:12.993128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:12.993607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:12.994077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:12.994906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:12.995205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:12.996579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.006387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 504/1000] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:13.007405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:13.059574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84f18-cc3b-4fa1-8a19-523ceb56a0f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.068436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:13.070690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:13.095506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.096173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.096456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.096734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.097331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.098772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:13.107973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:13.109322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:13.110251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 497/1001] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:13.116048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:13.120641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dd777-51e1-4eb4-8953-b6fa7682a491 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.157231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.157950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.158223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.158489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.159231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:13.159529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.160904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.171742 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 505/1002] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Sep 9 06:17:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:13.172733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:13.231371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac716a-cfb7-4985-a2c6-4163d18d7942 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.265943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.266471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.266751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.267025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.267686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:13.267991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.269476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.275425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 498/1003] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:13.280528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:13.285362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80786767-785f-4800-ba49-4bb60d7a581c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.309159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:13.322689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.323312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.323621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.323954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.324212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:13.324639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.326012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.345556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 506/1004] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:17:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:17:13.395257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3241-ed0f-4cc8-81c1-160c60f80dd1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.442942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.443896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.444322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.444833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.445685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:13.446131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.447685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.454234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 499/1005] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:13.460061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:13.575669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e781d16-80c1-45f4-a990-c6784dc96810 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.609944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.610558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.610832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.611097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.611813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] RBAC: Adding query filter params (interface=public, service_id=ea779b334c6e4d9e82e6ed664496a849, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:13.612118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.613483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df6acb-4282-4618-b8b7-1d5dc2737a55 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.620133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 507/1006] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:17:13 2022] GET /identity/v3/endpoints?service_id=ea779b334c6e4d9e82e6ed664496a849&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:13.626221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:13.727285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:13.732643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3de9e-a622-46ab-b0ff-5c7995b07d77 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:13.766779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033172ed-f76a-4fde-a8ae-862595633f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:13.767392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033172ed-f76a-4fde-a8ae-862595633f86 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:13.767699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033172ed-f76a-4fde-a8ae-862595633f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:13.767971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033172ed-f76a-4fde-a8ae-862595633f86 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:13.768584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033172ed-f76a-4fde-a8ae-862595633f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:13.769931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033172ed-f76a-4fde-a8ae-862595633f86 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:13.783651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 500/1007] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:17:13 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:17:14.914430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff43e-9a05-4c78-a990-cca81290c363 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:14.914996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff43e-9a05-4c78-a990-cca81290c363 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:14.914996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff43e-9a05-4c78-a990-cca81290c363 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:14.916533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 508/1008] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:14.921957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d06fd-42b6-467f-a47c-58d782db1f88 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:14.922357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d06fd-42b6-467f-a47c-58d782db1f88 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:14.922541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d06fd-42b6-467f-a47c-58d782db1f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:14.940638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e20d06fd-42b6-467f-a47c-58d782db1f88 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:14.941825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e20d06fd-42b6-467f-a47c-58d782db1f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:15.013015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 501/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:15.287590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ff43e-9a05-4c78-a990-cca81290c363 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:15.337206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:15.393903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ff43e-9a05-4c78-a990-cca81290c363 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:15.428538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899eb6-22d4-4f01-9b90-3b0ba1175138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:15.429166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899eb6-22d4-4f01-9b90-3b0ba1175138 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:15.429442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899eb6-22d4-4f01-9b90-3b0ba1175138 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:15.429713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899eb6-22d4-4f01-9b90-3b0ba1175138 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:15.430392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12899eb6-22d4-4f01-9b90-3b0ba1175138 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:15.431841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12899eb6-22d4-4f01-9b90-3b0ba1175138 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:15.452055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 509/1010] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:17:15 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:15.561681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c748f0-b6bf-44d7-a84f-368d7ffb4ce1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:15.561997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c748f0-b6bf-44d7-a84f-368d7ffb4ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:15.562236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c748f0-b6bf-44d7-a84f-368d7ffb4ce1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:15.563818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 502/1011] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:15.570314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977d70a-ee8b-4236-806b-90ac2160a025 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:15.570619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977d70a-ee8b-4236-806b-90ac2160a025 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:15.570866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977d70a-ee8b-4236-806b-90ac2160a025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:15.590206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d977d70a-ee8b-4236-806b-90ac2160a025 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:15.591432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d977d70a-ee8b-4236-806b-90ac2160a025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:15.672071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 510/1012] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:15.940254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c748f0-b6bf-44d7-a84f-368d7ffb4ce1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:15.945701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368cc97-9e3a-44e7-84b3-f19ae753eb75 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:15.946116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368cc97-9e3a-44e7-84b3-f19ae753eb75 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:15.946224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368cc97-9e3a-44e7-84b3-f19ae753eb75 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:15.947791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 511/1013] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:17:15.952680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:15.953221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:15.953497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:15.972442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:15.974193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:16.000905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:16.044535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:16.051420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 512/1014] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:16.054649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c748f0-b6bf-44d7-a84f-368d7ffb4ce1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:16.090871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:16.091830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:16.092322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:16.092829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:16.094132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:16.095822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:16.097893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 503/1015] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:16.103979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:16.205160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d957394-27e7-4536-b339-42f0c54b2e1a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:16.240044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:16.240658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:16.240936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:16.241210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:16.241853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:16.243284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:16.270005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 513/1016] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:17:16 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:16.323327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:16.436401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6905138-7bd9-4fdf-82e9-ca878d523af1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:16.474376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:16.474902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:16.475174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:16.475440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:16.476054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:16.477417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:16.486356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:16.487575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:16.488704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 504/1017] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:16.495505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:16.511973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:16.613784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23892f-8c8b-4a48-b124-bd088186170a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:16.649257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:16.649865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:16.650140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:16.650409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:16.651031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:16.651329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:16.652718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:16.662672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 514/1018] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:16.664287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:16.691807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:16.772213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee5bd3-c995-4a8f-be04-8aad842fd695 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:16.807893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:16.807893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:16.807893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:16.807893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:16.808314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:16.808396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:16.809761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e13fd3-9fa2-4d45-9a44-bb18f1aba323 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:16.814912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 505/1019] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Sep 9 06:17:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:16.820685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:16.901911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:16.925099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fbeae-c304-4fec-b603-5991b9b16c3a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:16.959377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2c227-3d1a-4399-bec2-8a52b1521a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:16.960106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c227-3d1a-4399-bec2-8a52b1521a91 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:16.960366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c227-3d1a-4399-bec2-8a52b1521a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:16.960537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c227-3d1a-4399-bec2-8a52b1521a91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:16.961206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2c227-3d1a-4399-bec2-8a52b1521a91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:16.962592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2c227-3d1a-4399-bec2-8a52b1521a91 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:16.982948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 515/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:17:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:17.651417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceb9bd-7c60-4aaf-8ac9-9c6a306ed211 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:17.651972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceb9bd-7c60-4aaf-8ac9-9c6a306ed211 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:17.651972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceb9bd-7c60-4aaf-8ac9-9c6a306ed211 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:17.653468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 506/1021] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:17.658445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:17.658737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:17.659012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:17.678483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:17.679682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:17.751577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 516/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:18.028797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceb9bd-7c60-4aaf-8ac9-9c6a306ed211 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:18.118359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:18.133610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceb9bd-7c60-4aaf-8ac9-9c6a306ed211 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:18.168013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:18.168637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.168911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.169179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.169757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:18.171107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:18.180229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:18.181255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:18.182166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:18.183708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 507/1023] 10.222.0.25 () {56 vars in 1105 bytes} [Fri Sep 9 06:17:18 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:18.187307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:18.247453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:18.290474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b9a8-8ac3-452d-a26a-1b6b6ace0b43 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:18.324129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:18.325022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.325500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.325915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.326697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] RBAC: Adding query filter params (name=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:18.327139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:18.328921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:18.339599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 517/1024] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:18.341612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:18.448503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77968c3d-a440-446c-9177-afdddab3072e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:18.483087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:18.483942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.484361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.484790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.485570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] RBAC: Adding query filter params (type=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:18.486011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:18.487595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2359-3306-4210-89a0-171625b7cd6c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:18.497820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 508/1025] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:18.499101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:18.510000 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c4a46-058b-4668-b84a-ccae8bbad893 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.510282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c4a46-058b-4668-b84a-ccae8bbad893 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.510553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c4a46-058b-4668-b84a-ccae8bbad893 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.512024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 509/1026] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:18.516872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.517155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.517430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.536197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:18.537265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:18.551962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:18.620939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 510/1027] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:18.621135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54226cab-4053-4fa3-aad9-30265e11a67e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:18.645862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:18.657455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:18.658063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.658341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.658617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.659440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] RBAC: Adding query filter params (interface=public, service_id=f672680eab714563b5514fb86ba91186, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:18.659784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:18.661672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:18.671399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 518/1028] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:17:18 2022] GET /identity/v3/endpoints?service_id=f672680eab714563b5514fb86ba91186&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:18.672600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:18.775556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4bc98-e21f-4a5c-9498-f6df2f043c9f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:18.811370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:18.811926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:18.812209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:18.812479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:18.813082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:18.814436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:18.827573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 511/1029] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:17:18 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:17:18.895584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:19.007134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0357af6-7a4c-4b68-9712-6c68f8f79a0c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:19.041904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d739b4-bb78-4ccf-9759-de753c50a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:19.042538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d739b4-bb78-4ccf-9759-de753c50a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:19.042811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d739b4-bb78-4ccf-9759-de753c50a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:19.043079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d739b4-bb78-4ccf-9759-de753c50a5c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:19.044227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d739b4-bb78-4ccf-9759-de753c50a5c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:19.045646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d739b4-bb78-4ccf-9759-de753c50a5c4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:19.051843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 519/1030] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:19.053452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:19.066093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:19.163530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d8c0-ddcd-4094-ad6d-1054d8099b88 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:19.198550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de0390-775f-40e1-be0a-b85bee6276fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:19.199165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de0390-775f-40e1-be0a-b85bee6276fc None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:19.199449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de0390-775f-40e1-be0a-b85bee6276fc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:19.199756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de0390-775f-40e1-be0a-b85bee6276fc None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:19.200409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de0390-775f-40e1-be0a-b85bee6276fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:19.201839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de0390-775f-40e1-be0a-b85bee6276fc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:19.225612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 512/1031] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:17:19 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:19.260263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344ed3b-a78b-4cb9-a9e9-73d7be73900c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:19.260916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344ed3b-a78b-4cb9-a9e9-73d7be73900c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:19.260916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344ed3b-a78b-4cb9-a9e9-73d7be73900c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:19.262493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 520/1032] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:19.268302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:19.268595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:19.268877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:19.289538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:19.290795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:19.360179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:19.375590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 513/1033] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:19.638368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344ed3b-a78b-4cb9-a9e9-73d7be73900c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:19.695572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:19.743986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344ed3b-a78b-4cb9-a9e9-73d7be73900c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:19.777996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:19.778622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:19.778896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:19.779163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:19.789974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:19.794750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:19.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:19.796638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:19.799889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 521/1034] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Sep 9 06:17:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:19.801728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:19.903150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a1aaa-491d-4a4e-828e-fff79085cc27 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:19.937928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:19.938542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:19.938821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:19.939092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:19.939794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:19.940099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:19.941456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:19.954592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 514/1035] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:17:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:19.956069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:20.063714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a1668-3b51-4a08-a549-994dfbcffcd0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:20.081427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:20.102211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:20.102854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:20.103133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:20.103405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:20.113715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:20.115151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:20.130217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:20.131183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:20.132471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:20.132471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:20.132471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:20.135844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 522/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:20.137085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:20.200885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:20.244524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a230677-b286-4a47-a24c-35ea7ac80114 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:20.278071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:20.278670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:20.278946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:20.279213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:20.279871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:20.280169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:20.281527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:20.293165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 515/1037] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:20.298543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:20.401791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158457c-865d-4c61-abde-a6e0a00c2027 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:20.435673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:20.436312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:20.436696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:20.436977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:20.445093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:20.446583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:17:20.458385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:20.459349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:20.464776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 523/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:17:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:20.467138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:20.576183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bdce9-f5bf-4c11-9129-5ac08c7ac273 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:20.595568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:20.615069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:20.615721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:20.616004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:20.616278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:20.617041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:20.617348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:20.618773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:20.637319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 516/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:17:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:20.642838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:20.672263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:20.747611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3eaac-4057-4548-a5ad-bf7275f0d343 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:20.780767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b95590-9df6-404a-9dc1-8cc3b3ef7caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:20.781372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b95590-9df6-404a-9dc1-8cc3b3ef7caf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:20.781650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b95590-9df6-404a-9dc1-8cc3b3ef7caf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:20.781919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b95590-9df6-404a-9dc1-8cc3b3ef7caf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:20.849500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b95590-9df6-404a-9dc1-8cc3b3ef7caf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:20.850944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b95590-9df6-404a-9dc1-8cc3b3ef7caf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:20.877144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:20.958424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 524/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:20.964031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.015800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86460a8d-b7fc-44b7-8185-c459a5ecb61e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.016097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86460a8d-b7fc-44b7-8185-c459a5ecb61e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.016267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86460a8d-b7fc-44b7-8185-c459a5ecb61e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.017617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 525/1041] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:17:21.023242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.023532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.023915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.041683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:21.042875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:21.066535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25383cc-6e05-4490-a0e1-875795a63b0c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.104408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.105044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.105418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.105620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.115396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 526/1042] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:21.120583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.122067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:21.137641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:21.138740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:21.140186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:21.140186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:21.140186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:21.143578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 517/1043] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:21.144776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.211982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:21.260989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb7864-6bb0-4ffe-898b-371bd13fc82b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.295168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.295732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.296010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.296279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.296918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:21.297221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.298573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.314976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 527/1044] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:21.316233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.346052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:21.390897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.425522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c5726-31c8-4886-a510-eff743cb5bcb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.461158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.461801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.462090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.462360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.469545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.470982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dc7a087c-6657-4191-b6e5-11a9e070764b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:17:21.483465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:21.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:21.485339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 518/1045] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:21.487216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b3307-d2d8-44eb-a332-0a9c9e9b2ce5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.487503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b3307-d2d8-44eb-a332-0a9c9e9b2ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.487809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b3307-d2d8-44eb-a332-0a9c9e9b2ce5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.489598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 519/1046] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:21.491132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b3307-d2d8-44eb-a332-0a9c9e9b2ce5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.497546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556e8a8-aded-460d-b021-0712e69cad02 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.514843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:21.532362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.533003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.533281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.533551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] PATH_INFO: `/v3/services/image` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.534130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.535513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e8295b92-4c02-43e5-954b-f480200ac533 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:21.545028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:21.546113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:21.547022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 528/1047] 10.222.0.25 () {56 vars in 1105 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:21.548425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.548425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.548425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.565396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:21.566590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:21.586260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:21.600459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b3307-d2d8-44eb-a332-0a9c9e9b2ce5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.633991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 529/1048] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:21.634251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.634808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.634912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.635218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.635433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.636164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:21.636469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.637893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.651714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 520/1049] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:21.656424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.736263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eadb2a-df13-4022-b44e-20c458d94f04 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.749660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:21.756148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd5013-df86-4b1a-95ba-d0bfc919568d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.773403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.774015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.774291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.774563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.775184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] RBAC: Adding query filter params (name=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:21.775485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.776876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.782410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 530/1050] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:21.788897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.796822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.797367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.797650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.797925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.798664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] RBAC: Adding query filter params (role.id=e1edcc38ef8d4a7982d84cff4e923c8d, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=dc56896d10094aed91db84ea2bf9dfb2) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:21.798974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.800488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.807588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 521/1051] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/role_assignments?user.id=dc56896d10094aed91db84ea2bf9dfb2&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=e1edcc38ef8d4a7982d84cff4e923c8d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:21.900508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6323b-61c3-41f1-b932-c414e6483904 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:21.909914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:21.927599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:21.941123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:21.942343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:21.942630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:21.943118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:21.943879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:21.944223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:21.945740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:21.952059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 531/1052] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:21.959097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:22.009709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:22.037475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8138-5e91-4541-8d0f-c4708b946b4f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:22.072669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceb619-019f-4ea2-a65a-127ee227d1c4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:22.074557 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:22.075204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:22.075483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:22.075787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:22.087823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:22.092688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:22.093705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:22.094568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 522/1053] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:22.100199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:22.110210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:22.111187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:22.111187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:22.111567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:22.112330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:22.114475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:22.136150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 532/1054] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:17:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:22.179051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:22.213798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06a70-50ab-4878-b9b2-1dd9654cea87 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:22.259605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:22.260231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:22.260510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:22.260782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:22.261463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:22.261769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:22.263163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:22.273646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 523/1055] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:22.279816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:22.391866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:22.399479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c807a-1bea-4083-9c01-7a5212030a69 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:22.435688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:22.436332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:22.436613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:22.436885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] PATH_INFO: `/v3/users/f648571c55f5494aaceba71ea59529f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:22.450881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] RBAC: Authorizing `identity:get_user(user_id=f648571c55f5494aaceba71ea59529f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:22.452356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:22.459359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 533/1056] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:22 2022] GET /identity/v3/users/f648571c55f5494aaceba71ea59529f5 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:22.460916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:22.567756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09e0b1-a54e-4ee0-b618-a8677634c88e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:22.600610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:22.607300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:22.607992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:22.608278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:22.608549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:22.679776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:22.681677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:22.786500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 524/1057] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:22.792348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:22.893804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e608b2-e08d-4c2c-84bb-08f26c0fd95c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:22.928524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e0afe-d7c3-4d56-9ee6-6f8e83849444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:22.929165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e0afe-d7c3-4d56-9ee6-6f8e83849444 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:22.929481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e0afe-d7c3-4d56-9ee6-6f8e83849444 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:22.929754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e0afe-d7c3-4d56-9ee6-6f8e83849444 None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:22.930916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e0afe-d7c3-4d56-9ee6-6f8e83849444 None admin] RBAC: Authorizing `identity:get_project(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:22.932314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e0afe-d7c3-4d56-9ee6-6f8e83849444 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:22.934092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 534/1058] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:22 2022] GET /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:22.940170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:22.970823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:23.046487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73c8af-2c18-4e05-8395-91b11ba4fbf7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:23.085680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:23.086286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:23.086563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:23.086830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:23.087567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d, user.id=f648571c55f5494aaceba71ea59529f5) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:23.087885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:23.089308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290ac40-bff6-404f-9ceb-acdc92362885 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:23.100114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 525/1059] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:22 2022] GET /identity/v3/role_assignments?user.id=f648571c55f5494aaceba71ea59529f5&scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:23.975367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2849-5088-42e0-94c7-3d10b2f4ffb5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:23.975963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2849-5088-42e0-94c7-3d10b2f4ffb5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:23.975963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2849-5088-42e0-94c7-3d10b2f4ffb5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:23.977321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 535/1060] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:23.982539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15fee-25f1-42d8-b40b-3c87fcdbaae5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:23.982824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15fee-25f1-42d8-b40b-3c87fcdbaae5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:23.983095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15fee-25f1-42d8-b40b-3c87fcdbaae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.001085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b6c15fee-25f1-42d8-b40b-3c87fcdbaae5 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:24.002261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b6c15fee-25f1-42d8-b40b-3c87fcdbaae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:24.071675 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 526/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:24.339827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2849-5088-42e0-94c7-3d10b2f4ffb5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:24.346520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0419-2215-446d-a15a-e6afada7989f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:24.346807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0419-2215-446d-a15a-e6afada7989f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:24.347074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0419-2215-446d-a15a-e6afada7989f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.349202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 527/1062] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:17:24.354435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:24.354768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:24.355045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.365816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:24.373213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:24.374399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:24.398210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:24.445642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2849-5088-42e0-94c7-3d10b2f4ffb5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:24.458184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 528/1063] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:24.487515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:24.488434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:24.489021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:24.489502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.499714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:24.504736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:24.505674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:24.506527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:24.508392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 536/1064] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Sep 9 06:17:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:24.514463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:24.620843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f20efb-f844-4a26-b0d4-f8850835c335 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:24.656765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:24.657381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:24.657664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:24.657939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.658610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:24.658915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:24.660818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:24.669326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 529/1065] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:17:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:24.675026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:24.736481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:24.757050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:24.762389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:24.781491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd38e5-0586-4cd8-9971-0cad67dca5ac None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:24.820101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:24.820737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:24.821011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:24.821279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.832248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:24.833474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:24.850147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:24.851162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:24.852211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:24.852211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:24.852211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:24.852211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 537/1066] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:24.852450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40838401-b9d1-4628-8c50-dd05121c1977 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:24.856793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:24.887854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:24.888410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:24.888689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:24.888965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:24.960208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:24.961657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:24.965884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610b6a3-49e1-4437-9951-752e61d24a76 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:24.978551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.002977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.003858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.003858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.004657 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.005572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:25.005744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.007078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.008642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.023191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 538/1067] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:25.028774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.069760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 530/1068] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 06:17:25.075440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.180042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f7f6b-528f-46fb-8df4-72ab3766a210 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.197394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.199183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b6d0-9bce-455d-9a6b-69d296d49b25 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.217929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.218968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.219428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.219900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.230838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.232564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:25.248814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:25.249993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:25.251110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:25.251110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:25.251110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:25.252612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 531/1069] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:25.257992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.259057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.259654 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.259758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.260145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.273685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.276073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:17:25.295909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:25.296985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:25.297946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 539/1070] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:25.304993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.366392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.397830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61367e8d-a945-47f6-ac6e-4d2f87e334c1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.420623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de7699-c8a8-45a8-aa68-32dd7875cf59 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.433907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.434459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.434741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.435015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.435700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:25.436004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.437367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12518e19-d289-4b9c-944a-8f235c020081 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.452240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 532/1071] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:25.452240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f761349-9165-4036-8696-10e7cb74f64c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.452240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f761349-9165-4036-8696-10e7cb74f64c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.452240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f761349-9165-4036-8696-10e7cb74f64c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.452240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 533/1072] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:25.463576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.463576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.463576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.463936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.463936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.463936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.463936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.464209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:25.464209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.464209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.479070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:25.480369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:25.483334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 540/1073] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:25.490593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.506193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.564209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 534/1074] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:25.611639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ba73c-265e-4ff0-9f18-07cd0e11713c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.626953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.653350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.653993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.654274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.654547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.665456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.666910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:25.683178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:25.684440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:25.685594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:25.685594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:25.685594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:25.685594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 541/1075] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:25.691073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.801754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3989f9-b143-4b67-aef1-dfa420371167 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.836122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.836357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.836735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.837021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.837295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.837941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:25.838245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:25.839624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:25.851447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 535/1076] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:25.857013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:25.875601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:25.939051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50e23-cffe-4766-8caf-a0a85df94e70 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.955947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c1c8-41fc-4a6f-8848-df56834cc48c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:25.972467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.973673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.973950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.974220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.984072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:25.989374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:25.990527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:25.991698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 542/1077] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:25.992588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:25.992885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:25.993218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:25.993497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:25.996463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:26.000182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.001620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:17:26.013162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:26.014390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:26.015499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 536/1078] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:26.020656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:26.025006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:26.098634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:26.099734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd6f5e-7325-457e-8ad1-985b6789d689 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:26.120596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944620b0-ab56-459a-aef3-a7f6fe7bec54 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:26.134253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:26.134859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:26.135139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:26.135414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:26.136101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:26.136404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.137761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.146264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 543/1079] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:26.152222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:26.154168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:26.154773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:26.155052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:26.155330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:26.156056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:26.156365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.157778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.170350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 537/1080] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:26.176982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:26.235211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:26.251755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bf377-02e6-4bb5-916c-056ff785f5d9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:26.275175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98dfc1-ec85-43dc-a061-358ad8e74961 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:26.286143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:26.286822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:26.287095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:26.287362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] PATH_INFO: `/v3/users/f648571c55f5494aaceba71ea59529f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:26.288586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=f648571c55f5494aaceba71ea59529f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.289997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.291854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 544/1081] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:26 2022] GET /identity/v3/users/f648571c55f5494aaceba71ea59529f5 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:26.297249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:26.309953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:26.310620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:26.310900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:26.311174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/dc56896d10094aed91db84ea2bf9dfb2/roles/e1edcc38ef8d4a7982d84cff4e923c8d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:26.313346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=dc56896d10094aed91db84ea2bf9dfb2, role_id=e1edcc38ef8d4a7982d84cff4e923c8d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.314760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.323516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:26.343578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 538/1082] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:17:26 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/dc56896d10094aed91db84ea2bf9dfb2/roles/e1edcc38ef8d4a7982d84cff4e923c8d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:17:26.404809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3b598-4dfe-4692-82c0-8461f25e3fe0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:26.439855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f78dd6-b188-43c6-9c12-6c2f80dfa908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:26.440505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f78dd6-b188-43c6-9c12-6c2f80dfa908 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:26.440781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f78dd6-b188-43c6-9c12-6c2f80dfa908 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:26.441052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f78dd6-b188-43c6-9c12-6c2f80dfa908 None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:26.442696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f78dd6-b188-43c6-9c12-6c2f80dfa908 None admin] RBAC: Authorizing `identity:get_project(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.444167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f78dd6-b188-43c6-9c12-6c2f80dfa908 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.445830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 545/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:26 2022] GET /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:26.451963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:26.559235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1aa6-b19a-4aa5-a5d2-a9779f86a232 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:26.596765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a97b0-4505-4211-b179-f8b8a9646cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:26.597418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a97b0-4505-4211-b179-f8b8a9646cbc None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:26.597745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a97b0-4505-4211-b179-f8b8a9646cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:26.598224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a97b0-4505-4211-b179-f8b8a9646cbc None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d/users/f648571c55f5494aaceba71ea59529f5/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:26.600346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a97b0-4505-4211-b179-f8b8a9646cbc None admin] RBAC: Authorizing `identity:create_grant(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d, user_id=f648571c55f5494aaceba71ea59529f5, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:26.601755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a97b0-4505-4211-b179-f8b8a9646cbc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:26.615484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 539/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:17:26 2022] PUT /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d/users/f648571c55f5494aaceba71ea59529f5/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:17:28.482449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9a20-b528-4490-9e5c-a7838a4ba4c3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:28.483026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9a20-b528-4490-9e5c-a7838a4ba4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:28.483026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9a20-b528-4490-9e5c-a7838a4ba4c3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:28.484459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 546/1085] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:28.489815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fe974-5463-401f-866e-9fbc459c0b37 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:28.490223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fe974-5463-401f-866e-9fbc459c0b37 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:28.490483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fe974-5463-401f-866e-9fbc459c0b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:28.508990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d91fe974-5463-401f-866e-9fbc459c0b37 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:28.510178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d91fe974-5463-401f-866e-9fbc459c0b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:28.562854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:28.581443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 540/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:28.732004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf9daf-9b70-4d29-a1c7-3c1282c5b296 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:28.732280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf9daf-9b70-4d29-a1c7-3c1282c5b296 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:28.732600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf9daf-9b70-4d29-a1c7-3c1282c5b296 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:28.734050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 547/1087] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:28.739175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:28.739479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:28.739780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:28.757909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:28.759081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:28.824220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 541/1088] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:28.855910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf9daf-9b70-4d29-a1c7-3c1282c5b296 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:28.880768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:28.960835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf9daf-9b70-4d29-a1c7-3c1282c5b296 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:28.994366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:28.994980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:28.995256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:28.995525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.006265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:29.011023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:29.012263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 06:17:29.013251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:29.015697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 548/1089] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Sep 9 06:17:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:29.018012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.099856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.116935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64eafc-59e1-49d1-9a25-79fd2f14f175 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.134930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.152390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.152998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.153279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.153546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.154207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:29.154508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.155894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.165480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 542/1090] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:29.171018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.185375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.203894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4208c-0ad6-4c49-9515-73e4d5711403 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.239511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.240164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.240440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.240711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.250754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:29.255957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:29.256988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:29.257838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 549/1091] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:29.262948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.274564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251d78-8060-4d46-bd1f-b07f885f8a6d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.308515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.309141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.309412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.309674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.320699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.322121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:29.337154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:29.338190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:29.339119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:29.339119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:29.339119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:29.339119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 543/1092] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:29.343951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.364841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e6674-fe0a-46af-b81a-b9227c00013a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.374341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.404316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.404925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.405205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.405473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.406129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:29.406427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.407994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.416026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 550/1093] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:29.422025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.425403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.445971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fcfe3-3e02-40fa-b3aa-251888ad5c7d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.479726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.480323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.480596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.480864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.481495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:29.481791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.483130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.494581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 544/1094] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:29.500700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.520347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0225c3-f60e-4656-86cc-a28fbe9b0064 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.554163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621403d-b264-472c-b159-962c900bc49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.554799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621403d-b264-472c-b159-962c900bc49d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.555077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621403d-b264-472c-b159-962c900bc49d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.555346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621403d-b264-472c-b159-962c900bc49d None admin] PATH_INFO: `/v3/users/f648571c55f5494aaceba71ea59529f5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.556495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621403d-b264-472c-b159-962c900bc49d None admin] RBAC: Authorizing `identity:get_user(user_id=f648571c55f5494aaceba71ea59529f5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.558394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621403d-b264-472c-b159-962c900bc49d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.564587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 551/1095] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/users/f648571c55f5494aaceba71ea59529f5 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:29.568483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621403d-b264-472c-b159-962c900bc49d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.593303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.602567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617ae49-576d-473d-b8e1-e3acf49c53f9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.636280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.636909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.637188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.637460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.639512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.646082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.647923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:17:29.659930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:29.661474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:29.662787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 545/1096] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:29.667627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.672069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621403d-b264-472c-b159-962c900bc49d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.705075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.705676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.705951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.706226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.766708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c50d9-a6f7-4642-b4a6-7ce88f2f8d7d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.772188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.773579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.801294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.801991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.802273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.802547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.803275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:29.803615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:29.805129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:29.822018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 546/1097] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:29.827678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.839384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.842818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:29.873956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 552/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 06:17:29.880290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:29.925958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336c704-4b97-4ad4-9228-bf9cad5b2796 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:29.962683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:29.963366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:29.963691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:29.963942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:29.978763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bba85-ccf7-42b9-b9bb-2453d6736786 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.013550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.014175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.014452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.014723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.015851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] RBAC: Authorizing `identity:get_project(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.017268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.018883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 553/1099] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:30.025463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:30.034657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.036481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.047500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:30.065951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:30.125342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628796-154c-4956-8ded-2c8f91422c46 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.136055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 547/1100] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 06:17:30.141844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:30.159810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.160436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.160813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.161087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.161802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d, user.id=f648571c55f5494aaceba71ea59529f5) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:30.162113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.163511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.183573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 554/1101] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:30 2022] GET /identity/v3/role_assignments?user.id=f648571c55f5494aaceba71ea59529f5&scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:30.246021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f809b10-ff75-4ed7-bd96-a5f293338c60 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.258287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:30.284037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.284592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.284868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.285183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.296307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.297789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:17:30.314061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:30.315202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:17:30.317487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:17:30.317487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:17:30.317487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:30.317487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 548/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:17:30.321877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:30.423349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14285-070d-4404-81dd-5c1e058b5090 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.426311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:30.458580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.459188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.459464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.459774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.460411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:30.460709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.462061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.473612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 555/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:30.479091 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:30.581461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919e214-260e-46c4-99e0-e8555d6a92d6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.615078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.615744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.616025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.616294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.622793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.624323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:17:30.635967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:30.636930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:30.637841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 549/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:17:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:30.642876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:30.740717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb310fe-3900-4710-a7a4-171a0b543dfd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.772816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.773418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.773693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.773964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.774644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:30.774953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.776395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e5edf-f761-4706-8b85-484a2a9b664d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.788045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 556/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:17:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:30.794055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:30.818650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:30.893132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4c03-99c5-4da2-8ef2-bf9344d7cb28 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:30.926246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:30.926841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:30.927115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:30.927383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:30.928138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] RBAC: Adding query filter params (role.id=e1edcc38ef8d4a7982d84cff4e923c8d, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=dc56896d10094aed91db84ea2bf9dfb2) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:30.928444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:30.929862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f824b-ffa7-4b3b-8101-6dcf38a9c5bc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:30.935778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 550/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:30 2022] GET /identity/v3/role_assignments?user.id=dc56896d10094aed91db84ea2bf9dfb2&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=e1edcc38ef8d4a7982d84cff4e923c8d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:32.363901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112edb2-3db7-4680-a830-795f533b9dc1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:32.364473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112edb2-3db7-4680-a830-795f533b9dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:32.364473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112edb2-3db7-4680-a830-795f533b9dc1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:32.365844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 557/1107] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:32.371059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:32.371494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:32.371772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:32.389965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:32.391133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:32.457624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 551/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:32.723808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112edb2-3db7-4680-a830-795f533b9dc1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:32.749529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:32.823564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112edb2-3db7-4680-a830-795f533b9dc1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:32.855519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a5afe-06a6-4272-b69a-78cd960c584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:32.856164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a5afe-06a6-4272-b69a-78cd960c584d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:32.856442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a5afe-06a6-4272-b69a-78cd960c584d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:32.856713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a5afe-06a6-4272-b69a-78cd960c584d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:32.857808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a5afe-06a6-4272-b69a-78cd960c584d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:32.859227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a5afe-06a6-4272-b69a-78cd960c584d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:32.860801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 558/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:32.867030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:32.884653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:32.965188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adfd89-335c-48f7-9a8e-2e2f40b44d2d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:32.999719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83d0b0-4239-4d92-87b5-41fa06549d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:33.000267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d0b0-4239-4d92-87b5-41fa06549d26 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:33.000549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d0b0-4239-4d92-87b5-41fa06549d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:33.000821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83d0b0-4239-4d92-87b5-41fa06549d26 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:33.001477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83d0b0-4239-4d92-87b5-41fa06549d26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:33.002901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83d0b0-4239-4d92-87b5-41fa06549d26 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:33.022825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 552/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:17:32 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:35.187805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e6574-6e8a-4009-9946-020ea6e0b981 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:35.188358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e6574-6e8a-4009-9946-020ea6e0b981 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:35.188358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e6574-6e8a-4009-9946-020ea6e0b981 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:35.189681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 559/1111] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:35.195681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:35.196080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:35.196231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:35.214860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:35.216070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:35.281766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 553/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:35.558669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e6574-6e8a-4009-9946-020ea6e0b981 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:35.650076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:35.659965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e6574-6e8a-4009-9946-020ea6e0b981 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:35.692816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:35.693437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:35.693715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:35.693986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] PATH_INFO: `/v3/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:35.695363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] RBAC: Authorizing `identity:get_role(role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:35.696752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:35.698062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 560/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:35 2022] GET /identity/v3/roles/8ace72a8232f495d8e03de411d3ec43b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:35.704153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:35.706313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:35.799988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ad450-c6cb-4f5c-bb8b-bd53127a381c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:35.833492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:35.834111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:35.834385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:35.834655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] PATH_INFO: `/v3/users/be84a44c4e3943938f1956ec26f51a5b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:35.847204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=be84a44c4e3943938f1956ec26f51a5b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:35.848648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:35.850208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 554/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:35 2022] GET /identity/v3/users/be84a44c4e3943938f1956ec26f51a5b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:35.856014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:35.957812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d16aab-5c43-46fb-a686-1d6496c990f0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:35.990571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:35.991171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:35.991450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:35.991760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:36.017587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:36.056784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:36.058158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:36.150981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 561/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:36.156907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:36.214970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:36.255266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235242ad-022e-4718-b5c7-2e73dae3f4d3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:36.288873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a80277-3bfc-4ded-8f9b-7e736314ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:36.289492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a80277-3bfc-4ded-8f9b-7e736314ddfd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:36.289765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a80277-3bfc-4ded-8f9b-7e736314ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:36.290031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a80277-3bfc-4ded-8f9b-7e736314ddfd None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:36.291121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a80277-3bfc-4ded-8f9b-7e736314ddfd None admin] RBAC: Authorizing `identity:get_project(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:36.292839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a80277-3bfc-4ded-8f9b-7e736314ddfd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:36.294426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 555/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:36 2022] GET /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:36.300652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:36.357707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:36.397981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b49d-4f65-48df-a9a1-cba0d145521b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:36.430817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:36.431413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:36.431726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:36.431998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:36.432710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] RBAC: Adding query filter params (role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d, user.id=be84a44c4e3943938f1956ec26f51a5b) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:36.433014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:36.434431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc889c95-4389-4be5-b645-697ca2b60189 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:36.439613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 562/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:36 2022] GET /identity/v3/role_assignments?user.id=be84a44c4e3943938f1956ec26f51a5b&scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:38.558782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee4b18-0edb-4b47-ba37-7b9a6bd07731 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:38.559202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee4b18-0edb-4b47-ba37-7b9a6bd07731 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:38.559333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee4b18-0edb-4b47-ba37-7b9a6bd07731 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:38.560728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 556/1118] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:38.566028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:38.566536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:38.566658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:38.584871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:38.586030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:38.650054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 563/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:38.916852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee4b18-0edb-4b47-ba37-7b9a6bd07731 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:39.016545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee4b18-0edb-4b47-ba37-7b9a6bd07731 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:39.042647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:39.051595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:39.052222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:39.052504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:39.052778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] PATH_INFO: `/v3/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:39.054167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] RBAC: Authorizing `identity:get_role(role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:39.055535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:39.056905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 557/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:38 2022] GET /identity/v3/roles/8ace72a8232f495d8e03de411d3ec43b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:39.062703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:39.115795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:39.158782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3ffb-5769-4e32-8cfe-739b25130492 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:39.190469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:39.191605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:39.192012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:39.192283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] PATH_INFO: `/v3/users/be84a44c4e3943938f1956ec26f51a5b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:39.193375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=be84a44c4e3943938f1956ec26f51a5b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:39.194781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:39.196347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 564/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:39 2022] GET /identity/v3/users/be84a44c4e3943938f1956ec26f51a5b => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:39.202034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:39.299354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9668b-2196-41f2-a4b4-c5d095c52e77 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:39.334427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abe1aa-33d8-4446-8288-154190cce11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:39.335054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe1aa-33d8-4446-8288-154190cce11e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:39.335337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe1aa-33d8-4446-8288-154190cce11e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:39.335638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe1aa-33d8-4446-8288-154190cce11e None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:39.336914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abe1aa-33d8-4446-8288-154190cce11e None admin] RBAC: Authorizing `identity:get_project(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:39.338344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abe1aa-33d8-4446-8288-154190cce11e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:39.339995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 558/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:39 2022] GET /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:39.345966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:39.443626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9ed02-d3dc-4e15-8256-de43808d6bd1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:39.465313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:39.477169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a53bb-f2e4-4922-9b05-d7a85e762717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:39.477816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a53bb-f2e4-4922-9b05-d7a85e762717 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:39.478095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a53bb-f2e4-4922-9b05-d7a85e762717 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:39.478369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a53bb-f2e4-4922-9b05-d7a85e762717 None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d/users/be84a44c4e3943938f1956ec26f51a5b/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:39.480490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a53bb-f2e4-4922-9b05-d7a85e762717 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d, user_id=be84a44c4e3943938f1956ec26f51a5b, role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:39.481902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a53bb-f2e4-4922-9b05-d7a85e762717 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:39.494281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 565/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:17:39 2022] PUT /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d/users/be84a44c4e3943938f1956ec26f51a5b/roles/8ace72a8232f495d8e03de411d3ec43b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:17:41.588893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4a7eb-f785-4735-b350-42ae4cde2084 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:41.589408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4a7eb-f785-4735-b350-42ae4cde2084 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:41.589511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4a7eb-f785-4735-b350-42ae4cde2084 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:41.590801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 559/1124] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:41.595898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:41.596303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:41.596454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:41.614796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:41.616014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:41.680517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 566/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:41.951447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4a7eb-f785-4735-b350-42ae4cde2084 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:42.013810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:42.052317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4a7eb-f785-4735-b350-42ae4cde2084 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:42.085112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:42.085729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:42.086003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:42.086269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] PATH_INFO: `/v3/roles/8ace72a8232f495d8e03de411d3ec43b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:42.087657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] RBAC: Authorizing `identity:get_role(role_id=8ace72a8232f495d8e03de411d3ec43b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:42.089005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:42.090291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 560/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:41 2022] GET /identity/v3/roles/8ace72a8232f495d8e03de411d3ec43b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:42.095984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:42.189500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82276a-6b6d-4b9c-a610-831ed68311e0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:42.202065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:42.223471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:42.224115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:42.224394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:42.224662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] PATH_INFO: `/v3/users/be84a44c4e3943938f1956ec26f51a5b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:42.225752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] RBAC: Authorizing `identity:get_user(user_id=be84a44c4e3943938f1956ec26f51a5b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:42.227154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:42.228716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 567/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:42 2022] GET /identity/v3/users/be84a44c4e3943938f1956ec26f51a5b => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:42.234385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:42.333281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bf20-93e6-4106-94b1-59ed84cb1ed0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:42.361921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:42.367225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:42.367865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:42.368140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:42.368408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:42.432974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:42.434396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:42.530492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 561/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:42.537045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:42.632473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cfe91-204d-4d67-a0f9-122613402699 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:42.664849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812361c-f3a4-4e9c-83d6-c23e7ea726e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:42.665472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812361c-f3a4-4e9c-83d6-c23e7ea726e8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:42.665752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812361c-f3a4-4e9c-83d6-c23e7ea726e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:42.666023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812361c-f3a4-4e9c-83d6-c23e7ea726e8 None admin] PATH_INFO: `/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:42.667122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812361c-f3a4-4e9c-83d6-c23e7ea726e8 None admin] RBAC: Authorizing `identity:get_project(project_id=ecebfe0d36f44ce18a71e3d2aa34d17d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:42.668568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812361c-f3a4-4e9c-83d6-c23e7ea726e8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:42.670168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 568/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:42 2022] GET /identity/v3/projects/ecebfe0d36f44ce18a71e3d2aa34d17d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:42.676980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:42.706614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:42.772919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683bb34-754e-40aa-83d1-0e5741bd679b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:42.805576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:42.806113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:42.806391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:42.806663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:42.807394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] RBAC: Adding query filter params (role.id=8ace72a8232f495d8e03de411d3ec43b, scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d, user.id=be84a44c4e3943938f1956ec26f51a5b) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:42.807731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:42.809144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53d35a-864c-492a-a8d8-0490d2653c27 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:42.814684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 562/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:42 2022] GET /identity/v3/role_assignments?user.id=be84a44c4e3943938f1956ec26f51a5b&scope.project.id=ecebfe0d36f44ce18a71e3d2aa34d17d&role.id=8ace72a8232f495d8e03de411d3ec43b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:45.092029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6e8b5-9d26-4e04-b33d-8c10e324e0dd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:45.092628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6e8b5-9d26-4e04-b33d-8c10e324e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:45.092628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6e8b5-9d26-4e04-b33d-8c10e324e0dd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:45.093890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 569/1131] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:45.099749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:45.100155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:45.100298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:45.118614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:45.119842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:45.185317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 563/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:45.456668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6e8b5-9d26-4e04-b33d-8c10e324e0dd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:45.505785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:45.556461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6e8b5-9d26-4e04-b33d-8c10e324e0dd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:45.590365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d5164-0f2f-4bdb-a5e5-fb63d7da72ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:45.590989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d5164-0f2f-4bdb-a5e5-fb63d7da72ec None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:45.591271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d5164-0f2f-4bdb-a5e5-fb63d7da72ec None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:45.591562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d5164-0f2f-4bdb-a5e5-fb63d7da72ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:45.592676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d5164-0f2f-4bdb-a5e5-fb63d7da72ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:45.594098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d5164-0f2f-4bdb-a5e5-fb63d7da72ec None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:45.595678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 570/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:45.602095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:45.628744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:45.698763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c2d2-44c3-4822-bfbc-f4910f8e4204 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:45.732704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb2bb4-ff5d-4a1b-93ef-32376bb800bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:45.733242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb2bb4-ff5d-4a1b-93ef-32376bb800bf None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:45.733519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb2bb4-ff5d-4a1b-93ef-32376bb800bf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:45.733787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb2bb4-ff5d-4a1b-93ef-32376bb800bf None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:45.734428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb2bb4-ff5d-4a1b-93ef-32376bb800bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:45.735870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb2bb4-ff5d-4a1b-93ef-32376bb800bf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:45.762689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 564/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:17:45 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:47.942567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa999f8-801d-4e19-9adc-82c736815eb6 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:47.943087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa999f8-801d-4e19-9adc-82c736815eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:47.943175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa999f8-801d-4e19-9adc-82c736815eb6 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:47.944503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 571/1135] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:47.949875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:47.950261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:47.950432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:47.969229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:47.970412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:48.035382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 565/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:48.305759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa999f8-801d-4e19-9adc-82c736815eb6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:48.403803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa999f8-801d-4e19-9adc-82c736815eb6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:48.419463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:48.437022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990befb-1e02-41b5-8702-c094f4d699cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:48.437648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990befb-1e02-41b5-8702-c094f4d699cc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:48.437929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990befb-1e02-41b5-8702-c094f4d699cc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:48.438204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990befb-1e02-41b5-8702-c094f4d699cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:48.439301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990befb-1e02-41b5-8702-c094f4d699cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:48.440850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990befb-1e02-41b5-8702-c094f4d699cc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:48.442399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 572/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:17:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:48.448199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:48.459144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:48.547255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e3c1a-897a-461d-b008-516eee69e4a0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:48.580466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac1356-8d7c-4f37-85b4-8f91d07b561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:48.581740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac1356-8d7c-4f37-85b4-8f91d07b561b None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:48.582058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac1356-8d7c-4f37-85b4-8f91d07b561b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:48.582334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac1356-8d7c-4f37-85b4-8f91d07b561b None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:48.582989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac1356-8d7c-4f37-85b4-8f91d07b561b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:48.584430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac1356-8d7c-4f37-85b4-8f91d07b561b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:48.605860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 566/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:17:48 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:17:50.789112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56a917-0542-4a32-a039-46152c82806e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:50.789639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56a917-0542-4a32-a039-46152c82806e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:50.789639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56a917-0542-4a32-a039-46152c82806e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:50.790994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 573/1139] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:50.796424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:50.796883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:50.797041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:50.815847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:50.817039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:50.873767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:50.883422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 567/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:51.167827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56a917-0542-4a32-a039-46152c82806e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:51.265452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56a917-0542-4a32-a039-46152c82806e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:51.299202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:51.300090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:51.300366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:51.300633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:51.310184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:51.314941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:51.316334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:51.317328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 574/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:17:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:51.322393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:51.420696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72612f71-62b7-4189-ab7e-c6249818e991 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:51.453576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:51.454179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:51.454463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:51.454735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:51.455402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:51.456283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:51.457637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:51.465552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 568/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:51.472373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:51.516484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:51.570397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ff562-d1f5-469b-80f3-9cbc259badc2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:51.604094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:51.604710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:51.604984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:51.605249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] PATH_INFO: `/v3/users/1c1a73f732824a85a0e9293f263093e7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:51.617653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] RBAC: Authorizing `identity:get_user(user_id=1c1a73f732824a85a0e9293f263093e7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:51.619072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:51.620839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 575/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:51 2022] GET /identity/v3/users/1c1a73f732824a85a0e9293f263093e7 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:51.626472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:51.681780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:51.725673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90772d58-f6d8-4771-b532-e9553ba97c3f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:51.759167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:51.759794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:51.760074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:51.760348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:51.825463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:51.827164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:51.879994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:51.922712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 569/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:51.929500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:52.026867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0ac56-15fb-4e5c-9e87-de15b75b18d3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:52.028332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:52.061547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47454187-ef50-4006-92b4-531e3c5de316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:52.062162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47454187-ef50-4006-92b4-531e3c5de316 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:52.062437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47454187-ef50-4006-92b4-531e3c5de316 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:52.062703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47454187-ef50-4006-92b4-531e3c5de316 None admin] PATH_INFO: `/v3/projects/f2f9d21cb3fa41269e3ec2c264724107` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:52.063815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47454187-ef50-4006-92b4-531e3c5de316 None admin] RBAC: Authorizing `identity:get_project(project_id=f2f9d21cb3fa41269e3ec2c264724107)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:52.065846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47454187-ef50-4006-92b4-531e3c5de316 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:52.067523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 576/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:51 2022] GET /identity/v3/projects/f2f9d21cb3fa41269e3ec2c264724107 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:52.073666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:52.168551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a0dee-4cce-4ce5-a3f4-98335e8fdd82 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:52.202300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:52.202897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:52.203173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:52.203441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:52.204192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=f2f9d21cb3fa41269e3ec2c264724107, user.id=1c1a73f732824a85a0e9293f263093e7) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:52.204497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:52.205943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8955d54-01f2-4cbc-90f8-2fddd6869513 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:52.211177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 570/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:52 2022] GET /identity/v3/role_assignments?user.id=1c1a73f732824a85a0e9293f263093e7&scope.project.id=f2f9d21cb3fa41269e3ec2c264724107&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:17:54.319139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76521f-3593-4e4a-8142-de10a8c87426 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:54.319687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76521f-3593-4e4a-8142-de10a8c87426 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:54.319687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76521f-3593-4e4a-8142-de10a8c87426 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:54.321045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 577/1147] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:54.326068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:54.326474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:54.326673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:54.344560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:54.345733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:54.359336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:54.411635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 571/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:17:54.681728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76521f-3593-4e4a-8142-de10a8c87426 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:54.781227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76521f-3593-4e4a-8142-de10a8c87426 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:54.813012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:54.813620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:54.813895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:54.814161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:54.824505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:54.829503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:54.831166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:17:54.832384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 578/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:54.837194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:54.932550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa4c4-89b7-4cb5-be8d-1208a5fd2c3b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:54.965540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:54.966132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:54.966405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:54.966670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:54.967327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:54.967648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:54.968995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:54.977463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 572/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:54.983099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:55.013312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:55.081737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73388dd2-7ecf-439c-a5eb-ba88d6e11515 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:55.114895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:55.115506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:55.115812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:55.116087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] PATH_INFO: `/v3/users/1c1a73f732824a85a0e9293f263093e7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:55.117183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] RBAC: Authorizing `identity:get_user(user_id=1c1a73f732824a85a0e9293f263093e7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:55.118663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:55.120235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 579/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:54 2022] GET /identity/v3/users/1c1a73f732824a85a0e9293f263093e7 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:55.125974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:55.140758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:55.223827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a41df-0886-4d6c-8083-c8c50bca7777 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:55.255148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9081538-daf6-4fc5-8df0-da4d768cda6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:55.255791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9081538-daf6-4fc5-8df0-da4d768cda6c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:55.256066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9081538-daf6-4fc5-8df0-da4d768cda6c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:55.256331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9081538-daf6-4fc5-8df0-da4d768cda6c None admin] PATH_INFO: `/v3/projects/f2f9d21cb3fa41269e3ec2c264724107` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:55.257418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9081538-daf6-4fc5-8df0-da4d768cda6c None admin] RBAC: Authorizing `identity:get_project(project_id=f2f9d21cb3fa41269e3ec2c264724107)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:55.258827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9081538-daf6-4fc5-8df0-da4d768cda6c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:55.260440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 573/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:55 2022] GET /identity/v3/projects/f2f9d21cb3fa41269e3ec2c264724107 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:55.266302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:55.361017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:55.365379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b666-1310-4f43-98ba-d8fb61f13a15 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:55.399089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90884b-254b-4319-b2f0-9c78ed8333e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:55.399771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90884b-254b-4319-b2f0-9c78ed8333e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:55.400046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90884b-254b-4319-b2f0-9c78ed8333e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:55.400321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90884b-254b-4319-b2f0-9c78ed8333e6 None admin] PATH_INFO: `/v3/projects/f2f9d21cb3fa41269e3ec2c264724107/users/1c1a73f732824a85a0e9293f263093e7/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:55.402208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90884b-254b-4319-b2f0-9c78ed8333e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2f9d21cb3fa41269e3ec2c264724107, user_id=1c1a73f732824a85a0e9293f263093e7, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:55.403654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90884b-254b-4319-b2f0-9c78ed8333e6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:55.416712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 580/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:17:55 2022] PUT /identity/v3/projects/f2f9d21cb3fa41269e3ec2c264724107/users/1c1a73f732824a85a0e9293f263093e7/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:17:57.539129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d9f6e-6115-4ba2-b8a4-44fe2e2a5e20 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:57.539787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d9f6e-6115-4ba2-b8a4-44fe2e2a5e20 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:57.539787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d9f6e-6115-4ba2-b8a4-44fe2e2a5e20 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:57.541047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 574/1154] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:17:57.546467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:57.546840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:57.546964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:57.566279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:17:57.567487 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:17:57.632548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 581/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:17:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:17:57.910694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d9f6e-6115-4ba2-b8a4-44fe2e2a5e20 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:57.991995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:58.011616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d9f6e-6115-4ba2-b8a4-44fe2e2a5e20 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:58.044671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:58.045291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:58.045569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:58.045840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:58.056929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:17:58.061733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:17:58.063729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:17:58.064755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 575/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:17:58.068959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:58.152908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:58.165761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec09c6b-ab92-4770-8978-ba57baec2dd4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:58.198305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:58.198903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:58.199176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:58.199441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:58.200118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:58.200415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:58.202018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:58.210536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 582/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:17:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:58.216445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:58.313577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9867c5-017c-42df-991c-0f6926bc2455 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:58.346884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:58.347497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:58.347808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:58.348082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] PATH_INFO: `/v3/users/1c1a73f732824a85a0e9293f263093e7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:58.349201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] RBAC: Authorizing `identity:get_user(user_id=1c1a73f732824a85a0e9293f263093e7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:58.350611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:58.352199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 576/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:17:58 2022] GET /identity/v3/users/1c1a73f732824a85a0e9293f263093e7 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:58.357875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:58.454544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390cc1e-8bb6-4cd5-a432-2e47cd6422b9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:58.487483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:58.488100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:58.488376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:58.488644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:58.506805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:58.554160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:58.555632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:58.651953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 583/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:17:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:17:58.657922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:58.669182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:58.755666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98fbe4-6a7d-4b70-8cfc-50b651cecd78 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:58.788695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c3db1-956f-4cd1-94ce-86206b4e7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:58.789311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c3db1-956f-4cd1-94ce-86206b4e7f78 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:58.789587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c3db1-956f-4cd1-94ce-86206b4e7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:58.789858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c3db1-956f-4cd1-94ce-86206b4e7f78 None admin] PATH_INFO: `/v3/projects/f2f9d21cb3fa41269e3ec2c264724107` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:58.790954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c3db1-956f-4cd1-94ce-86206b4e7f78 None admin] RBAC: Authorizing `identity:get_project(project_id=f2f9d21cb3fa41269e3ec2c264724107)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:58.792409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c3db1-956f-4cd1-94ce-86206b4e7f78 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:58.794012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 577/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:17:58 2022] GET /identity/v3/projects/f2f9d21cb3fa41269e3ec2c264724107 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:17:58.801002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:17:58.849565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:17:58.897469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21537c-aa98-4ab2-9503-fccd776eb674 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:17:58.930539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:17:58.931129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:17:58.931406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:17:58.931705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:17:58.932414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=f2f9d21cb3fa41269e3ec2c264724107, user.id=1c1a73f732824a85a0e9293f263093e7) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:17:58.932717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:17:58.934150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81185b2-0050-48ad-83a1-f0f0debfd92b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:17:58.940024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 584/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:17:58 2022] GET /identity/v3/role_assignments?user.id=1c1a73f732824a85a0e9293f263093e7&scope.project.id=f2f9d21cb3fa41269e3ec2c264724107&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:18:01.095030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe56e00-826a-48c4-9818-d6c27af9dcd0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:01.096125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe56e00-826a-48c4-9818-d6c27af9dcd0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:01.096125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe56e00-826a-48c4-9818-d6c27af9dcd0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:01.097481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 578/1162] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:18:01.102753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:01.103151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:01.103301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:01.122142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:01.123328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:01.187613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 585/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:18:01.460689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe56e00-826a-48c4-9818-d6c27af9dcd0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:01.537653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:01.561936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe56e00-826a-48c4-9818-d6c27af9dcd0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:01.594377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:01.594980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:01.595260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:01.596070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:01.661524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:01.662943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:01.734029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:01.759044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 579/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:18:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:18:01.765023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:01.808704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:01.860389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29c01-4748-43f0-b7ed-26827a00d40b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:01.892854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:01.893498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:01.893775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:01.894046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:01.900522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:01.901946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8c2eb83b-0554-4a96-b8fb-984c7503958b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:18:01.913606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:18:01.914815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:18:01.915749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 586/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:18:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:18:01.920720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:02.016470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d851-4b07-46be-85bf-c381341267c2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:02.050113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:02.050710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:02.050989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:02.051260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:02.051930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:18:02.052235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:02.053588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eadda4-3b51-4daa-9044-ec5209691139 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:02.060546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 580/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Sep 9 06:18:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:04.187393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eea2e5-9818-42df-bd60-c70363235761 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:04.187955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eea2e5-9818-42df-bd60-c70363235761 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:04.187955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eea2e5-9818-42df-bd60-c70363235761 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:04.189405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 587/1167] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:18:04.194811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998f6d-ba0d-4e14-a0e9-cb2bdc1113d9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:04.195216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998f6d-ba0d-4e14-a0e9-cb2bdc1113d9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:04.195361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998f6d-ba0d-4e14-a0e9-cb2bdc1113d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:04.214598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1b998f6d-ba0d-4e14-a0e9-cb2bdc1113d9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:04.215808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b998f6d-ba0d-4e14-a0e9-cb2bdc1113d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:04.249448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:04.281591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 581/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:18:04.550319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eea2e5-9818-42df-bd60-c70363235761 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:04.654447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eea2e5-9818-42df-bd60-c70363235761 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:04.657707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:04.690716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66621b89-c0e3-4cb5-8c7c-2d4b3c145098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:04.691338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621b89-c0e3-4cb5-8c7c-2d4b3c145098 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:04.691650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621b89-c0e3-4cb5-8c7c-2d4b3c145098 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:04.691930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66621b89-c0e3-4cb5-8c7c-2d4b3c145098 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:04.692605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621b89-c0e3-4cb5-8c7c-2d4b3c145098 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:04.693965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66621b89-c0e3-4cb5-8c7c-2d4b3c145098 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:04.711264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 588/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Fri Sep 9 06:18:04 2022] POST /identity/v3/domains => generated 267 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:18:06.885323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78158771-57a8-421b-a9c6-42a77fffd049 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:06.885845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78158771-57a8-421b-a9c6-42a77fffd049 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:06.885845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78158771-57a8-421b-a9c6-42a77fffd049 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:06.887138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 582/1170] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:18:06.893120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:06.893529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:06.893662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:06.911354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:06.912560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:06.976197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 589/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:18:07.244892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78158771-57a8-421b-a9c6-42a77fffd049 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:07.343795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78158771-57a8-421b-a9c6-42a77fffd049 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:07.376552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac7968-4b57-4ccc-a0b2-5eea976f184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:07.377170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7968-4b57-4ccc-a0b2-5eea976f184e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:07.377442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7968-4b57-4ccc-a0b2-5eea976f184e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:07.377711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7968-4b57-4ccc-a0b2-5eea976f184e None admin] PATH_INFO: `/v3/domains/f068a1a6a66a417397e3afb3cf9d09d8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:07.387772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac7968-4b57-4ccc-a0b2-5eea976f184e None admin] RBAC: Authorizing `identity:get_domain(domain_id=f068a1a6a66a417397e3afb3cf9d09d8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:07.389200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac7968-4b57-4ccc-a0b2-5eea976f184e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:07.390745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 583/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Fri Sep 9 06:18:07 2022] GET /identity/v3/domains/f068a1a6a66a417397e3afb3cf9d09d8 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:07.396786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:07.491770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:07.499267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7494c-098a-44e1-98b6-30d92309c37b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:07.532425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c8d4c-1e3f-47a1-904b-10d05d113993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:07.533807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8d4c-1e3f-47a1-904b-10d05d113993 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:07.534104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8d4c-1e3f-47a1-904b-10d05d113993 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:07.534374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8d4c-1e3f-47a1-904b-10d05d113993 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:07.535020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c8d4c-1e3f-47a1-904b-10d05d113993 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:07.536451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c8d4c-1e3f-47a1-904b-10d05d113993 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:07.561929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 590/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:18:07 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:18:09.757382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91f53e-ec2e-4b69-88e8-b7212bdf6de9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:09.757946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91f53e-ec2e-4b69-88e8-b7212bdf6de9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:09.757946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91f53e-ec2e-4b69-88e8-b7212bdf6de9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:09.759255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 584/1174] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:18:09.764509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:09.764929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:09.765035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:09.783703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:09.784867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:09.848542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 591/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:18:10.125242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91f53e-ec2e-4b69-88e8-b7212bdf6de9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:10.155784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:10.226366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91f53e-ec2e-4b69-88e8-b7212bdf6de9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:10.257820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b4ef5-2b44-46bf-9eb1-5572988160c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:10.258431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b4ef5-2b44-46bf-9eb1-5572988160c8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:10.258705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b4ef5-2b44-46bf-9eb1-5572988160c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:10.258970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b4ef5-2b44-46bf-9eb1-5572988160c8 None admin] PATH_INFO: `/v3/domains/f068a1a6a66a417397e3afb3cf9d09d8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:10.260085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b4ef5-2b44-46bf-9eb1-5572988160c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f068a1a6a66a417397e3afb3cf9d09d8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:10.261496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b4ef5-2b44-46bf-9eb1-5572988160c8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:10.263023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 585/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Fri Sep 9 06:18:10 2022] GET /identity/v3/domains/f068a1a6a66a417397e3afb3cf9d09d8 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:10.269360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:10.342436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:10.366022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0aef81-ba20-4a38-b4ed-e6b691f38b48 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:10.398852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966ce0e-1332-43b8-a15f-bb5d770e3f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:10.399450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ce0e-1332-43b8-a15f-bb5d770e3f97 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:10.399826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ce0e-1332-43b8-a15f-bb5d770e3f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:10.400102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ce0e-1332-43b8-a15f-bb5d770e3f97 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:10.400750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966ce0e-1332-43b8-a15f-bb5d770e3f97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:10.402166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966ce0e-1332-43b8-a15f-bb5d770e3f97 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:10.424723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 592/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:18:10 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:18:12.615761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094cb2-86c2-4f7b-99da-1849687a3147 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:12.616258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094cb2-86c2-4f7b-99da-1849687a3147 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:12.616348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094cb2-86c2-4f7b-99da-1849687a3147 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:12.617633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 586/1178] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:18:12.622885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:12.623361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:12.623455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:12.642162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:12.643312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:12.707913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 593/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:18:12.978341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094cb2-86c2-4f7b-99da-1849687a3147 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:13.074172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:13.079735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094cb2-86c2-4f7b-99da-1849687a3147 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:13.112981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:13.113594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:13.113870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:13.114139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:13.124234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:18:13.129950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:18:13.131094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:18:13.132191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 587/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:18:13.137088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:13.195245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:13.232758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911560d2-4f35-4a06-a8ab-f6d75ee4f78c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:13.266040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:13.266670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:13.267012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:13.267285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:13.267968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:18:13.268269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:13.269618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:13.277306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 594/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:18:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:18:13.283480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:13.380742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd050-b375-4f8f-bd56-5e32f0052089 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:13.412858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:13.413475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:13.413755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:13.414028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] PATH_INFO: `/v3/users/443c062a7f1d408191c43e3adb0641c9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:13.426588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] RBAC: Authorizing `identity:get_user(user_id=443c062a7f1d408191c43e3adb0641c9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:13.428047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:13.429622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 588/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:18:13 2022] GET /identity/v3/users/443c062a7f1d408191c43e3adb0641c9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:13.435355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:13.540842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67c4f2-d717-426c-b8bd-f1dd2bb48c3c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:13.563178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:13.575045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:13.575682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:13.575960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:13.576232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:13.641554 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:13.643778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:13.736952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 595/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:18:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:18:13.742899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:13.761098 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:13.842603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f6e78-4846-41b9-8d96-ddfcce75a77b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:13.875735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb4a4d-908f-4ecd-b67e-3ad86ef4bb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:13.876412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb4a4d-908f-4ecd-b67e-3ad86ef4bb40 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:13.876690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb4a4d-908f-4ecd-b67e-3ad86ef4bb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:13.876962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb4a4d-908f-4ecd-b67e-3ad86ef4bb40 None admin] PATH_INFO: `/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:13.878075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb4a4d-908f-4ecd-b67e-3ad86ef4bb40 None admin] RBAC: Authorizing `identity:get_project(project_id=499d69e4eb1f4bf88fd59125a2a11d80)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:13.879498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb4a4d-908f-4ecd-b67e-3ad86ef4bb40 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:13.881128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 589/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:18:13 2022] GET /identity/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:13.887206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:13.911810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:13.983227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66de10-d404-4a03-980b-16158cb144e4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:14.014803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:14.015387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:14.015694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:14.015964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:14.017204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=499d69e4eb1f4bf88fd59125a2a11d80, user.id=443c062a7f1d408191c43e3adb0641c9) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:18:14.017509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:14.018970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee32fb-a907-42ee-a9aa-55e73b236a00 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:14.024250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 596/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:18:13 2022] GET /identity/v3/role_assignments?user.id=443c062a7f1d408191c43e3adb0641c9&scope.project.id=499d69e4eb1f4bf88fd59125a2a11d80&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:16.148873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f440e6-9289-4154-a8b9-e54671f193eb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:16.149427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f440e6-9289-4154-a8b9-e54671f193eb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:16.149427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f440e6-9289-4154-a8b9-e54671f193eb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:16.150763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 590/1186] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:18:16.156007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:16.156408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:16.156609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:16.174801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:16.175992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:16.239404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 597/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:18:16.510628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f440e6-9289-4154-a8b9-e54671f193eb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:16.602265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:16.619471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f440e6-9289-4154-a8b9-e54671f193eb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:16.651860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:16.652481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:16.652760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:16.653031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:16.663500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:18:16.668354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:18:16.669495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:18:16.670754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 591/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:18:16.676269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:16.693643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:16.771393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb09a06-bf60-436d-99a3-81cce52a65f4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:16.804181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:16.804771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:16.805044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:16.805313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:16.805959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:18:16.806257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:16.807638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:16.815489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 598/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:18:16.821282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:16.931061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049691ac-e983-49f3-9e3d-909189e5083d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:16.965347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:16.965959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:16.966235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:16.966504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] PATH_INFO: `/v3/users/443c062a7f1d408191c43e3adb0641c9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:16.967637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] RBAC: Authorizing `identity:get_user(user_id=443c062a7f1d408191c43e3adb0641c9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:16.969047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:16.971319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 592/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:18:16 2022] GET /identity/v3/users/443c062a7f1d408191c43e3adb0641c9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:16.977166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:17.065404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:17.084866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87211d-6cef-42b3-8af3-07f067fb774d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:17.117103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c1503-102a-452c-bd67-e7351cd0d9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:17.117724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c1503-102a-452c-bd67-e7351cd0d9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:17.118003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c1503-102a-452c-bd67-e7351cd0d9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:17.118276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c1503-102a-452c-bd67-e7351cd0d9c3 None admin] PATH_INFO: `/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:17.119379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c1503-102a-452c-bd67-e7351cd0d9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=499d69e4eb1f4bf88fd59125a2a11d80)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:17.120829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c1503-102a-452c-bd67-e7351cd0d9c3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:17.122428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 599/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:18:16 2022] GET /identity/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:18:17.128886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:17.143920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:17.227434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd524ad3-65f1-4473-b27f-9c20c49db975 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:17.261004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae1530-7b30-4a59-9dee-1a172d84f829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:17.261650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1530-7b30-4a59-9dee-1a172d84f829 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:17.261932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1530-7b30-4a59-9dee-1a172d84f829 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:17.262207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1530-7b30-4a59-9dee-1a172d84f829 None admin] PATH_INFO: `/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80/users/443c062a7f1d408191c43e3adb0641c9/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:17.264121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae1530-7b30-4a59-9dee-1a172d84f829 None admin] RBAC: Authorizing `identity:create_grant(project_id=499d69e4eb1f4bf88fd59125a2a11d80, user_id=443c062a7f1d408191c43e3adb0641c9, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:17.265541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae1530-7b30-4a59-9dee-1a172d84f829 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:17.278398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 593/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:18:17 2022] PUT /identity/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80/users/443c062a7f1d408191c43e3adb0641c9/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:18:19.401513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea294-85ad-4ba0-bd3e-b2e2238d7853 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:19.402042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea294-85ad-4ba0-bd3e-b2e2238d7853 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:19.402042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea294-85ad-4ba0-bd3e-b2e2238d7853 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:19.403376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 600/1193] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:18:19.408866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:19.409291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:19.409417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:19.427119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:18:19.428361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:18:19.492853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 594/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:18:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:18:19.758726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea294-85ad-4ba0-bd3e-b2e2238d7853 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:19.856083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea294-85ad-4ba0-bd3e-b2e2238d7853 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:19.888944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:19.889564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:19.889845 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:19.890118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:19.901194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:18:19.906023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:18:19.907212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:18:19.908189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 601/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:18:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:18:19.913017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:19.915193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:20.010993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da466ab5-f080-43a8-be88-b97b1fee4dbf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:20.043650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:20.044241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:20.044514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:20.044779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:20.045433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:18:20.045729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:20.047078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:20.055354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 595/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:18:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:20.061235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:20.072983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:20.159243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8278ff-c84c-4a52-aeb2-de35836d7e74 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:20.191678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:20.192294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:20.192570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:20.192843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] PATH_INFO: `/v3/users/443c062a7f1d408191c43e3adb0641c9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:20.193948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] RBAC: Authorizing `identity:get_user(user_id=443c062a7f1d408191c43e3adb0641c9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:20.195362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:20.196941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 602/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Sep 9 06:18:20 2022] GET /identity/v3/users/443c062a7f1d408191c43e3adb0641c9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:20.202851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:20.263866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:20.301356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de2958-6d07-48d2-81f2-d4cb2b5b4e0a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:20.333216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:20.333805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:20.334078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:20.334345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:20.398722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:20.400175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:20.457582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:20.495664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 596/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:18:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:18:20.501738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:20.576427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:20.599740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f60ed3-dcfa-473f-bb86-34c54216b732 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:20.631646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aea967-0b2e-4761-85d3-43702190af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:20.632265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea967-0b2e-4761-85d3-43702190af22 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:20.632542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea967-0b2e-4761-85d3-43702190af22 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:20.632814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea967-0b2e-4761-85d3-43702190af22 None admin] PATH_INFO: `/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:20.633912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aea967-0b2e-4761-85d3-43702190af22 None admin] RBAC: Authorizing `identity:get_project(project_id=499d69e4eb1f4bf88fd59125a2a11d80)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:20.635328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aea967-0b2e-4761-85d3-43702190af22 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:20.636955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 603/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Sep 9 06:18:20 2022] GET /identity/v3/projects/499d69e4eb1f4bf88fd59125a2a11d80 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:20.643590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:18:20.739128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15d8c5-cee5-4a11-babf-3f854f6ccc4e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:18:20.771695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:18:20.772281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:18:20.772555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:18:20.772821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:18:20.773537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=499d69e4eb1f4bf88fd59125a2a11d80, user.id=443c062a7f1d408191c43e3adb0641c9) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:18:20.773839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:18:20.775237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0b23-fc2c-4096-bf2a-9e79ea394f0d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:18:20.780807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 597/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:18:20 2022] GET /identity/v3/role_assignments?user.id=443c062a7f1d408191c43e3adb0641c9&scope.project.id=499d69e4eb1f4bf88fd59125a2a11d80&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:18:30.628068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=28 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:18:30.778264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:16.505621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eedea-16ff-4cee-b780-9b7241ca5295 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:16.506228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eedea-16ff-4cee-b780-9b7241ca5295 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:16.506228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eedea-16ff-4cee-b780-9b7241ca5295 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:16.507486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 604/1201] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:16.513791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:16.514286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:16.514616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:16.539018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:16.540486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:16.643538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 598/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:19:16.935973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eedea-16ff-4cee-b780-9b7241ca5295 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:17.059339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eedea-16ff-4cee-b780-9b7241ca5295 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:17.101803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:17.102497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:17.102944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:17.103218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:17.119679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:17.121730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:17.141420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:17.142531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:17.143892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:17.143892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:17.143892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:17.143892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 605/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:17.149013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:17.310518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea4226-8b38-42bc-8ec5-91399dc97187 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:17.355462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:17.366241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:17.366855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:17.367132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:17.367404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:17.368160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:17.368604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:17.370012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3417766-0a7b-4633-86d3-7b7b327e43cb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:17.383400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 599/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:17.389783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:17.456232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:17.526992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd29f7-463d-4ab1-992e-224ce0e90ea1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:17.573347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeffd09-280a-4e8b-ab9b-8ce50e905664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:17.574327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffd09-280a-4e8b-ab9b-8ce50e905664 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:17.574621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffd09-280a-4e8b-ab9b-8ce50e905664 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:17.575180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffd09-280a-4e8b-ab9b-8ce50e905664 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:17.576623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeffd09-280a-4e8b-ab9b-8ce50e905664 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:17.578351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeffd09-280a-4e8b-ab9b-8ce50e905664 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:17.610862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 606/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Sep 9 06:19:17 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:19:20.025302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836652ae-6056-4c43-a143-d18b8c5c3594 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:20.025984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836652ae-6056-4c43-a143-d18b8c5c3594 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:20.025984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836652ae-6056-4c43-a143-d18b8c5c3594 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:20.027234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 600/1206] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:20.033477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:20.033956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:20.034120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:20.056073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:20.057435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:20.137336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 607/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:20.461040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836652ae-6056-4c43-a143-d18b8c5c3594 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:20.585840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836652ae-6056-4c43-a143-d18b8c5c3594 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:20.629007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:20.629647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:20.630106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:20.630223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:20.641605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:19:20.646438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:20.647466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:20.648374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 601/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:20.654097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:20.701029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:20.766439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f665c-86a4-4b1c-88cc-fe17036f2296 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:20.802774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:20.803383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:20.803702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:20.803972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:20.804630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:20.804947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:20.806400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:20.820539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 608/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:19:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:20.821880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:20.881544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:20.937561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3053e-ad5e-4d50-af46-a8c026fd6e84 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:20.975229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:20.975902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:20.976179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:20.976444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:20.987408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:20.988978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:21.005549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:21.006653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:21.013483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:21.013483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:21.013483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:21.013483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 602/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:21.013748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:21.128866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:21.131811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ce1ee-6e7e-472e-b72b-484cef3ee2d4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:21.176715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:21.176715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:21.176715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:21.176715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:21.177219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:21.177313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:21.178773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:21.192054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 609/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:21.197748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:21.310853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967bbe-81d2-438c-8ad4-d7e4c1ab4156 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:21.313886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:21.351914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:21.352677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:21.353022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:21.353362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:21.361386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:21.362903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:19:21.375258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:21.376566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:21.377855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:21.383517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 603/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:21.384475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:21.501722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b261d-ba81-4e9c-b863-e05da41be08c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:21.539519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:21.540162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:21.540438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:21.540712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:21.541467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:21.541871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:21.543374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:21.572310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 610/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Sep 9 06:19:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:21.573304 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:21.689423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ecb67-8952-4ddf-adbd-0b7c600e8a28 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:21.727702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:21.728487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:21.728859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:21.729110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:21.772809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:21.814798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:21.816389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:21.942374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 604/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:19:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:19:21.943915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:21.968108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:22.061972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a76a0-3b89-4a3a-a43e-2b718a051f5d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:22.113138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:22.113869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:22.114277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:22.114579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:22.128058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:22.129550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:22.148609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:22.150074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:22.151273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:22.151273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:22.151273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:22.155354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 611/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:22.156864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:22.231146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:22.282210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236aa15-aecd-42c8-9c8a-017302fa5060 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:22.322753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:22.323382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:22.323723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:22.323956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:22.324855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:22.325664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:22.327134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:22.339337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 605/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:22.346011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:22.429351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:22.468396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e154a49-ae81-4531-b330-bce5b2c4a5e0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:22.507896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:22.508548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:22.508830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:22.509147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:22.518180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:22.519772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:19:22.532690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:22.534004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:22.535167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 612/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:19:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:22.544830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:22.681519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f3507-95d7-42d1-aeff-e5c3138de7dc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:22.706177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:22.720113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:22.720766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:22.721078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:22.721432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:22.722226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:22.722535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:22.724206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9bf9-25b6-4ec9-b49d-1d0e0ab2c852 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:22.739054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 606/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:19:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:22.745435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:22.852055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d103a6-2f2c-4626-bd72-97daa6bccb0e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:22.885880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:22.892134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:22.892738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:22.893015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:22.893303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:22.894003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=31e2bef8256c40c68bc7711c5922289d) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:22.894308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:22.895754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006d33c-5b3f-4d4a-b144-b69ae2d80dc5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:22.907261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 613/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:19:22 2022] GET /identity/v3/role_assignments?user.id=31e2bef8256c40c68bc7711c5922289d&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:25.068899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de2bfc-e4ba-43a7-87dc-7a7f7c233973 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:25.069588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de2bfc-e4ba-43a7-87dc-7a7f7c233973 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:25.069588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de2bfc-e4ba-43a7-87dc-7a7f7c233973 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:25.070730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 607/1220] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:25.076706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:25.077178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:25.077526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:25.097846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:25.099059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:25.177391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 614/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:25.451103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de2bfc-e4ba-43a7-87dc-7a7f7c233973 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:25.572242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de2bfc-e4ba-43a7-87dc-7a7f7c233973 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:25.611471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:25.612136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:25.612409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:25.612677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:25.623559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:19:25.628449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:25.629995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:25.631190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 608/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:19:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:25.635832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:25.737788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4e839-4ad1-4527-9bbd-afafcf51720f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:25.761877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:25.773964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:25.774577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:25.774856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:25.775121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:25.775813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:25.776113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:25.777478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:25.786221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 615/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:19:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:25.792031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:25.838074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:25.901396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d1687-8537-4c6c-81e1-4a0792f054dd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:25.948624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:25.949287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:25.950349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:25.950598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:25.961603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:25.963132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:25.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:25.982197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:25.983208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:25.983208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:25.983208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:25.983208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 609/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:25.988837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:26.105893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd97cb-a0ef-460b-9c89-04e8d7069b98 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:26.142953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:26.143627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:26.143943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:26.144214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:26.144904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:26.145230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:26.146583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:26.163085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 616/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:26.169725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:26.271510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:26.279041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf5936-95f9-4878-8f1b-0d3c3116ce52 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:26.315531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:26.316183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:26.316466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:26.316739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:26.325935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:26.327533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:19:26.339666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:26.340807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:26.346853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 610/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:26.347639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:26.398568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:26.467601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e109ba-f27e-4d3d-8bab-de0a0eacfa96 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:26.505151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:26.505843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:26.506115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:26.506380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:26.507094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:26.507393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:26.509076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:26.527237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 617/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Sep 9 06:19:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:26.533252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:26.645149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a86c6-36e6-4c38-a6d4-d338981fa207 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:26.680837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:26.681476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:26.681774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:26.682029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:26.694189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:26.695684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:26.710720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:26.711935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:26.713031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:26.713031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:26.713031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:26.713031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 611/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:26.718333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:26.824778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89c90c-8878-41ed-ae13-3d7fe3f8ed5e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:26.827497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:26.862479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:26.863087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:26.863361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:26.863689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:26.864341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:26.864640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:26.866471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:26.879720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 618/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:26.886450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:26.896800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:27.006541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a37cc-4e77-4d80-96d5-d6d5b41e2bcf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:27.041629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:27.042355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:27.042725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:27.042953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:27.051961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:27.053668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:19:27.066650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:27.067899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:27.069081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 612/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:19:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:27.074110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:27.192523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6835507-1d80-4595-acb7-6889c6c718c2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:27.230150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:27.230754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:27.231170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:27.231448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:27.232162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:27.232464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:27.233886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2abf4-707b-438e-a720-ada603acb9a6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:27.248959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 619/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:19:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:27.255254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:27.327585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:27.367371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119b2fb-12bc-4190-b31e-cb24e7b9dd46 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:27.406524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4f5be-9d61-4891-b6b2-cc4e5e1db85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:27.407445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4f5be-9d61-4891-b6b2-cc4e5e1db85a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:27.407948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4f5be-9d61-4891-b6b2-cc4e5e1db85a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:27.408180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4f5be-9d61-4891-b6b2-cc4e5e1db85a None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/31e2bef8256c40c68bc7711c5922289d/roles/d0c0fe5816e24c43aaf708d42a9557aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:27.424227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4f5be-9d61-4891-b6b2-cc4e5e1db85a None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=31e2bef8256c40c68bc7711c5922289d, role_id=d0c0fe5816e24c43aaf708d42a9557aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:27.425632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4f5be-9d61-4891-b6b2-cc4e5e1db85a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:27.438195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 613/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:19:27 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/31e2bef8256c40c68bc7711c5922289d/roles/d0c0fe5816e24c43aaf708d42a9557aa => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:19:29.684214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387e27-a97c-46a2-a06a-bc085bce935f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:29.684817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387e27-a97c-46a2-a06a-bc085bce935f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:29.684817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387e27-a97c-46a2-a06a-bc085bce935f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:29.686083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 620/1233] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:29.691365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:29.691827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:29.691927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:29.711786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:29.714343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:29.792726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 614/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 06:19:30.111936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08387e27-a97c-46a2-a06a-bc085bce935f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:30.142096 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:30.227702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08387e27-a97c-46a2-a06a-bc085bce935f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:30.263847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:30.264549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:30.264867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:30.265173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:30.277319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:19:30.283174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:30.284470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:30.285414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 621/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Sep 9 06:19:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:30.290811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:30.339817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:30.400564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f32f7c-7b1d-4f77-a8da-dfa0ef08298e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:30.437956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:30.438769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:30.439079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:30.439441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:30.440137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:30.440438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:30.441795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:30.452445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 615/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Sep 9 06:19:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:30.459018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:30.559359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:30.582021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0a7e1-2c46-40de-a402-4ed0767bf904 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:30.624223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:30.624857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:30.625139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:30.625414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:30.636193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:30.637918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:30.654593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:30.662763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:30.663977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:30.663977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:30.663977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:30.663977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 622/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:30.664471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:30.771750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f474872-3fc7-41b4-afb5-8d946603c5a4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:30.781897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:30.810856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:30.811496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:30.811811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:30.812087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:30.812769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:30.813077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:30.814466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:30.828072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 616/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:30.836654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:30.946650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153f469-a1db-4a70-9e35-90e8a441b309 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:30.992446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:30.993029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:30.993294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:30.993561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:30.996342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:31.002308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:31.003858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:19:31.015652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:31.016859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:31.018250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 623/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:31.023468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:31.139002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611cfc18-54f3-4773-9d1b-123c61f287eb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:31.176698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:31.177371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:31.177670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:31.178000 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:31.178738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:31.179046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:31.180684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:31.200379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 617/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Sep 9 06:19:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:31.206100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:31.323557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564135d7-3ded-4ddd-8c29-152eb40bbe2d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:31.360709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:31.361347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:31.361702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:31.361989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:31.433985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:31.441977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:31.443435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:31.561107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 624/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:19:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:19:31.568857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:31.599463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:31.681689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377411d0-a737-4c5a-8966-99da0a38accd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:31.721224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:31.721945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:31.722351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:31.722688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:31.734895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:31.736596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:31.753209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:31.754458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:31.755391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:31.755391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:31.755391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:31.755391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 618/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:31.760531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:31.865934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:31.873333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f3d89-d057-4d47-b7e4-7c4d8488af42 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:31.912736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:31.913470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:31.913815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:31.914114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:31.914900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:31.915199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:31.916882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:31.931913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 625/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:31.939035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:32.036811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:32.070648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c21be-416f-4746-91a1-7afefced921f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:32.107921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:32.108771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:32.109189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:32.109549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:32.117334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:32.118681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:19:32.131106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:32.132426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:32.133538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 619/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:19:32.138643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:32.256458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef257e-d0e9-4dfd-bd03-3ffa1d32a382 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:32.295753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:32.296473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:32.296819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:32.297147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:32.297855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:32.298150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:32.299611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecce84-a42d-4995-bd6f-4330f723b933 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:32.311707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 626/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:19:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:32.318038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:32.438696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f5c21-6029-4217-b07e-faaeeae82afb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:32.477898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:32.478515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:32.478792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:32.479069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:32.479862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] RBAC: Adding query filter params (role.id=d0c0fe5816e24c43aaf708d42a9557aa, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=31e2bef8256c40c68bc7711c5922289d) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:32.480176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:32.482336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d3c77-9406-4a37-8e7d-70217fe9d8fa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:32.488687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 620/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:19:32 2022] GET /identity/v3/role_assignments?user.id=31e2bef8256c40c68bc7711c5922289d&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=d0c0fe5816e24c43aaf708d42a9557aa => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:35.040712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb411c-bde1-49b1-8db2-300bda5ebb64 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:35.041281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb411c-bde1-49b1-8db2-300bda5ebb64 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:35.041392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb411c-bde1-49b1-8db2-300bda5ebb64 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:35.042676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 627/1247] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:35.048163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:35.048609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:35.048722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:35.055229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:35.070799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:35.072042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:35.144770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 621/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:35.411486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb411c-bde1-49b1-8db2-300bda5ebb64 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:35.430560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:35.518431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb411c-bde1-49b1-8db2-300bda5ebb64 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:35.563915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:35.565057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:35.565549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:35.565834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:35.582363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:19:35.587343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:35.589202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:19:35.590272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:35.594071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 628/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:19:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:35.595975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:35.709604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b43cc-276c-42ac-8034-668bb5813777 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:35.743486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:35.750283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:35.750920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:35.751202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:35.751477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:35.752215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:35.752523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:35.753927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:35.768662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 622/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:35.770067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:35.853073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:35.878207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27734-ef1a-415d-98ea-d55714b59649 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:35.914491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:35.915142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:35.915422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:35.915717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:35.927237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:35.929340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:35.947516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:35.948650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:35.950750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:35.950750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:35.950750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:35.950750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 629/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:35.956022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:36.077700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f183ec6-8529-46e0-8354-e1541e7ee690 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:36.115961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:36.116584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:36.116857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:36.117122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:36.117777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:36.118075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:36.119448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:36.136470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 623/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:36.142766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:36.244253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0b09b-bf29-4528-bb04-1d634f093c08 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:36.271822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:36.280410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:36.281054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:36.281320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:36.281595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:36.289312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:36.290788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:19:36.302566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:36.303803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:36.304978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 630/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:36.310082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:36.367152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:36.414122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3ef18-bfcf-415c-9c7d-139b775aa1b3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:36.446975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:36.447602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:36.447884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:36.448154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:36.448879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:36.449743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:36.451243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:36.468293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 624/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Sep 9 06:19:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:36.474133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:36.570509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49472196-811c-44cc-a93f-5538ec46948d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:36.603076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:36.603711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:36.603985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:36.604252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:36.656861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:36.669302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:36.670816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:36.764955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 631/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:19:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 06:19:36.770723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:36.866048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0db24-47f9-4cd6-9118-bfcda0f84acd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:36.879014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:36.900070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:36.900689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:36.900964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:36.901233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:36.910905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:36.912305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:36.926744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:36.928023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:36.929284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:36.929284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:36.929284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:36.929284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 625/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:36.934289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:37.030302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:37.042423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad9140-4b20-4988-9049-9b4f1117d3b6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:37.076388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:37.076988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:37.077265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:37.077534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:37.078166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:37.078465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:37.079852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:37.090913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 632/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:37.096644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:37.195086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658c0f5-3c0d-496b-b0f0-52bdb5df978f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:37.227715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:37.228338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:37.228618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:37.228890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:37.235982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:37.237415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:19:37.249012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:37.250079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:37.251191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 626/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:37.255939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:37.351908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d14910-e5ea-43c1-a7d7-a6c89309219b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:37.385110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:37.385710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:37.385987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:37.386259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:37.386962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:37.387264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:37.388704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625609da-6dfe-481e-93b3-70f5cd0720bb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:37.400191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 633/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:37.406924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:37.432439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:37.506361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06ac32-06a0-43d4-823b-edea94946e9e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:37.538687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:37.539276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:37.539578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:37.539849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:37.540567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=31e2bef8256c40c68bc7711c5922289d) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:37.540873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:37.542292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7db682-91ce-4963-9e76-906e20571a72 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:37.548704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 627/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:19:37 2022] GET /identity/v3/role_assignments?user.id=31e2bef8256c40c68bc7711c5922289d&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:39.669847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb826b34-e18a-46ad-8c53-7f9449f7f3f3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:39.670392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb826b34-e18a-46ad-8c53-7f9449f7f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:39.670392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb826b34-e18a-46ad-8c53-7f9449f7f3f3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:39.671747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 634/1261] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:39.676990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:39.677399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:39.677540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:39.696207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:39.697400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:39.762106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 628/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:40.054689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb826b34-e18a-46ad-8c53-7f9449f7f3f3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:40.066184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:40.158121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb826b34-e18a-46ad-8c53-7f9449f7f3f3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:40.191432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:40.192087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:40.192362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:40.192632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:40.202705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:19:40.207514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:40.208508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:40.209556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 635/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:40.214653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:40.233334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:40.312687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d093a-ca86-423a-b971-a4cd09401e1f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:40.345231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:40.345824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:40.346101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:40.346371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:40.347027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:40.347324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:40.348713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:40.357341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 629/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:40.363066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:40.429591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:40.462416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c3f77-171e-4037-9bc5-66e9cc439de2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:40.496094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:40.496716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:40.496996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:40.497270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:40.506950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:40.508427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:40.522756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:40.523906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:40.524843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:40.524843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:40.524843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:40.524843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 636/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:40.530155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:40.612547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:40.631560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552fbcb-6335-47b3-a3ee-6395eab6b881 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:40.664697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:40.665302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:40.665583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:40.665858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:40.666501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:40.666804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:40.668747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:40.680097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 630/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:40.685821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:40.782892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab49b00-1120-4f89-b2fc-6e660f1a7cea None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:40.801814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:40.815866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:40.816482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:40.816762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:40.817035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:40.824718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:40.826133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:19:40.837041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:40.838130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:40.839124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 637/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:40.843873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:40.943763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68226286-bb1d-4250-bb43-fe25d73f6194 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:40.978158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:40.978753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:40.979029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:40.979303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:40.980048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:40.980504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:40.981938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:40.990282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:40.999186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 631/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Sep 9 06:19:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:41.004952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:41.101566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911c3d-0a19-4eaa-b890-ffb8f3e49da2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:41.133892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:41.134504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:41.134781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:41.135049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:41.145965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:41.147386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:41.161933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:41.163186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:41.165490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:41.165490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:41.165490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:41.165490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 638/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:41.169652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:41.266433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd49d53-6ef9-4742-9c40-c18f7aa77a0f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:41.299366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:41.299990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:41.300261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:41.300526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:41.301155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:41.301449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:41.302794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:41.314116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 632/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:41.319757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:41.351072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:41.418001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbc517-9b5b-43d3-9e21-de4ed39f7242 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:41.450926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:41.451574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:41.451857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:41.452133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:41.458612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:41.460090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:19:41.471276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:41.472347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:41.473317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 639/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:19:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:41.479168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:41.535804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:41.577840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41df8-282a-4759-92af-b33aae4e720c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:41.610391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:41.610993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:41.611271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:41.611573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:41.612258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:41.612563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:41.613970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29235c1f-4045-4583-acca-2b218bb9b517 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:41.626016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 633/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:19:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:41.631987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:41.720447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:41.729088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488c634-e007-446e-937e-0a9325471d4f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:41.762079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee72a0-03fd-4187-bff9-4805d225d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:41.762720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee72a0-03fd-4187-bff9-4805d225d5c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:41.762997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee72a0-03fd-4187-bff9-4805d225d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:41.763271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee72a0-03fd-4187-bff9-4805d225d5c2 None admin] PATH_INFO: `/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/31e2bef8256c40c68bc7711c5922289d/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:41.765144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee72a0-03fd-4187-bff9-4805d225d5c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=63076cf4a77649a2bdc642cd4f18db4e, user_id=31e2bef8256c40c68bc7711c5922289d, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:41.766580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee72a0-03fd-4187-bff9-4805d225d5c2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:41.778370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 640/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Sep 9 06:19:41 2022] PUT /identity/v3/projects/63076cf4a77649a2bdc642cd4f18db4e/users/31e2bef8256c40c68bc7711c5922289d/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:19:43.900813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbf69-3145-4a22-94f3-edf83be5677c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:43.901335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbf69-3145-4a22-94f3-edf83be5677c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:43.901460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cbf69-3145-4a22-94f3-edf83be5677c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:43.903025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 634/1274] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:43.908123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:43.908525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:43.908675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:43.926860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:43.928198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:44.008178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 641/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:44.280164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cbf69-3145-4a22-94f3-edf83be5677c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:44.390259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cbf69-3145-4a22-94f3-edf83be5677c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:44.409416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:44.428548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:44.429177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:44.429457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:44.429730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:44.442172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 06:19:44.447173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:44.451620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:44.452772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 635/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Sep 9 06:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:44.455412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:44.541549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:44.567643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8c2c-36d2-4598-8fbc-c114b07b2c99 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:44.611866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:44.612472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:44.612751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:44.613718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:44.614387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:44.614693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:44.616090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:44.624216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 642/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Sep 9 06:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:44.630061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:44.740986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf7ed7-3a41-4f0e-8911-dec6716aa8df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:44.775140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:44.775793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:44.776071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:44.776339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:44.787177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:44.788642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:44.804951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:44.806412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:44.807798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:44.807798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:44.807798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:44.807798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 636/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:44.812520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:44.920289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac096-4f6f-4354-8b87-d7285eea2ed7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:44.956922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:44.957534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:44.957811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:44.958084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:44.959011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:44.959579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:44.961346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:44.973191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 643/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:44.979057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:45.009527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:45.079161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e8f1-72ed-4d38-bc1b-f93c3bddb812 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:45.112610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:45.113239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:45.113520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:45.113794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:45.122680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:45.124139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 06:19:45.135143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:45.136880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:45.137869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 637/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:45.142604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:45.146318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:45.239279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90096d-036f-4e80-8996-a2bd1d092a65 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:45.272623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:45.273222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:45.273500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:45.273771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:45.274488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:45.274793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:45.276236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:45.293302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 644/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Sep 9 06:19:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:45.299268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:45.388960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:45.399204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d1967-31db-43df-b7a1-07172f712a8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:45.431905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:45.432502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:45.432808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:45.433661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:45.499047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:45.500508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:45.586334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:45.596718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 638/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Sep 9 06:19:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 06:19:45.602461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:45.663395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:45.701626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004171e-67c4-4b4e-b737-50a482fe5dde None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:45.734967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:45.735608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:45.735889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:45.736160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:45.745753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:45.747187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 06:19:45.761203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:45.762525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 06:19:45.763611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 06:19:45.763611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 06:19:45.763611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:45.763611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 645/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:45.767904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:45.863982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84388799-36e0-495a-917e-306f4825d9ad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:45.897013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:45.897607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:45.897885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:45.898156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:45.898791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:45.899093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:45.900476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:45.911949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 639/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Sep 9 06:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:45.918136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:46.015942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96890185-7bd5-4e75-ac91-fcf19e69b4f8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:46.029618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:46.049297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:46.049911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:46.050186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:46.050453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:46.056777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:46.058237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 06:19:46.068985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:46.070068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Sep 09 06:19:46.071199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 646/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Sep 9 06:19:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 06:19:46.076892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:46.126840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:46.176164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b27290-bcad-43bf-b310-a19ef95b42df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:46.208938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:46.209542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:46.209821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:46.210093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:46.210779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:46.211083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:46.212529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031a048-6863-45ba-b28c-3db471ea8968 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:46.224674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 640/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Sep 9 06:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:46.230807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:46.327711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8a8a-669d-414e-a01c-797f0171d493 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:46.359801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:46.360394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:46.360681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:46.360957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:46.361669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] RBAC: Adding query filter params (role.id=835f58ed7a1644f6975d2ad388d5b638, scope.project.id=63076cf4a77649a2bdc642cd4f18db4e, user.id=31e2bef8256c40c68bc7711c5922289d) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:46.361978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:46.363386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61340e51-9298-4799-9d18-8c13579361eb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:46.368953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 647/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Sep 9 06:19:46 2022] GET /identity/v3/role_assignments?user.id=31e2bef8256c40c68bc7711c5922289d&scope.project.id=63076cf4a77649a2bdc642cd4f18db4e&role.id=835f58ed7a1644f6975d2ad388d5b638 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:48.500999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea039c1f-8fc5-4d0c-8e84-8c8ca89730ef None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:48.501533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea039c1f-8fc5-4d0c-8e84-8c8ca89730ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:48.501533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea039c1f-8fc5-4d0c-8e84-8c8ca89730ef None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:48.502873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 641/1288] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:48.507971 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:48.508396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:48.508513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:48.526088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:48.527285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:48.555913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:48.592834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 648/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:48.857276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea039c1f-8fc5-4d0c-8e84-8c8ca89730ef None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:48.964739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea039c1f-8fc5-4d0c-8e84-8c8ca89730ef None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:48.978268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:49.007271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:49.007848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:49.008140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:49.008392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:49.008988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:49.010382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:49.021235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:49.022504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:49.023600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 642/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:19:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:49.028371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:49.124207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125fb51-5f89-4937-8c53-c898ddc1a38b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:49.156426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:49.157016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:49.157291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:49.157562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:49.158176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] RBAC: Adding query filter params (name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:49.158474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:49.159849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473afd9-2af6-434a-8000-1162856d93db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:49.164427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 649/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Sep 9 06:19:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:49.170871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:49.268604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5c3ce-1e0b-4ffb-9e2a-a3f60dc43ab1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:49.301001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:49.301590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:49.301871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:49.302146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:49.302772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:49.303078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:49.304460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c379afa-5b4c-44c2-be66-20d34f23f73d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:49.309327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 643/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Sep 9 06:19:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:51.455444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49b64e-8144-42cd-93ab-f1f3be72977e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:51.456030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49b64e-8144-42cd-93ab-f1f3be72977e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:51.456030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49b64e-8144-42cd-93ab-f1f3be72977e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:51.457354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 650/1293] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:51.462412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5b276-a8e6-4e53-8edd-636706f246cf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:51.462797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5b276-a8e6-4e53-8edd-636706f246cf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:51.462955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5b276-a8e6-4e53-8edd-636706f246cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:51.481547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b7d5b276-a8e6-4e53-8edd-636706f246cf None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:51.482711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7d5b276-a8e6-4e53-8edd-636706f246cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:51.511569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:51.548262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 644/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:51.819206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49b64e-8144-42cd-93ab-f1f3be72977e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:51.849833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:51.918637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49b64e-8144-42cd-93ab-f1f3be72977e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:51.957163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd2e35-177d-460e-a43b-4494a02fac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:51.957843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2e35-177d-460e-a43b-4494a02fac0a None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:51.958195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2e35-177d-460e-a43b-4494a02fac0a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:51.958542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2e35-177d-460e-a43b-4494a02fac0a None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:51.959334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd2e35-177d-460e-a43b-4494a02fac0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:51.961031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd2e35-177d-460e-a43b-4494a02fac0a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:51.978200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 651/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Sep 9 06:19:51 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:19:54.103013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d21474-6eee-432b-97ae-64ff70127b84 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:54.103595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d21474-6eee-432b-97ae-64ff70127b84 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:54.103595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d21474-6eee-432b-97ae-64ff70127b84 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:54.105050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 645/1296] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:54.110202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a8182-175c-4131-93da-10ff7407628c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:54.110597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a8182-175c-4131-93da-10ff7407628c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:54.110750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a8182-175c-4131-93da-10ff7407628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:54.128894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-759a8182-175c-4131-93da-10ff7407628c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:54.130081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-759a8182-175c-4131-93da-10ff7407628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:54.201388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 652/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:54.467366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d21474-6eee-432b-97ae-64ff70127b84 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:54.566874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d21474-6eee-432b-97ae-64ff70127b84 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:54.600389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:54.601014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:54.601296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:54.601570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:54.602155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:54.603510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:54.612852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:54.614025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:54.614904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 646/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:19:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:54.619904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a8182-175c-4131-93da-10ff7407628c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:54.638386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:54.719629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a8182-175c-4131-93da-10ff7407628c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:54.752584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:54.753177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:54.753452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:54.753720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:54.754335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:54.754635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:54.756015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:54.761361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 653/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Sep 9 06:19:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:54.767145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:54.798930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:54.866242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d907a2-b959-4006-b05f-2a56f9fa74c4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:54.898352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:54.898941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:54.899216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:54.899485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:54.900199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] RBAC: Adding query filter params (interface=public, service_id=c404d5dc023142929e673400c6b8ce37, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:54.900501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:54.902651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d3226-ebae-449d-9cec-4e4a0ea660a9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:54.907451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 647/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Sep 9 06:19:54 2022] GET /identity/v3/endpoints?service_id=c404d5dc023142929e673400c6b8ce37&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:19:54.913028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:55.002863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:55.021158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795e5c8-3a3e-43cf-93cc-3cd526511d90 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:55.053699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaccefd-fb7e-44e4-8df4-ce86b1106384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:55.054294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaccefd-fb7e-44e4-8df4-ce86b1106384 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:55.054572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaccefd-fb7e-44e4-8df4-ce86b1106384 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:55.054843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaccefd-fb7e-44e4-8df4-ce86b1106384 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:55.055439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaccefd-fb7e-44e4-8df4-ce86b1106384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:55.056819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaccefd-fb7e-44e4-8df4-ce86b1106384 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:55.064035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 654/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Sep 9 06:19:54 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:19:57.229010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529bd0e-6742-4b84-ac71-7c2a49c9b984 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:57.229633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529bd0e-6742-4b84-ac71-7c2a49c9b984 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:57.230408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529bd0e-6742-4b84-ac71-7c2a49c9b984 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:57.231896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 648/1302] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:19:57.237210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:57.237665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:57.237769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:57.255614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:19:57.256790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:19:57.320669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 655/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:19:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 06:19:57.597029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529bd0e-6742-4b84-ac71-7c2a49c9b984 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:57.692235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:57.699623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529bd0e-6742-4b84-ac71-7c2a49c9b984 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:57.732442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:57.733061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:57.733344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:57.733620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] PATH_INFO: `/v3/services/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:57.734207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:57.735587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 06:19:57.744273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 06:19:57.746085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Sep 09 06:19:57.747146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 649/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Sep 9 06:19:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 06:19:57.752016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:57.826651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:19:57.848444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda81a6d-3c39-4daf-9176-dfd787adcc82 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:57.881008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:57.881604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:57.881884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:57.882156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:57.882777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:19:57.883076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:57.884462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4dbee-be95-48be-8573-7872a83f7562 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:57.889992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 656/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Sep 9 06:19:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:19:57.895766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:19:57.994268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b02ae-a992-48ea-a89e-21abc489c2ec None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:19:58.027380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607026aa-dfbe-407c-a235-f8225c8fadc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:19:58.028004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607026aa-dfbe-407c-a235-f8225c8fadc8 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:19:58.028285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607026aa-dfbe-407c-a235-f8225c8fadc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:19:58.028556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607026aa-dfbe-407c-a235-f8225c8fadc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:19:58.029224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607026aa-dfbe-407c-a235-f8225c8fadc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:19:58.030577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607026aa-dfbe-407c-a235-f8225c8fadc8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:19:58.049035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 650/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:19:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:20:07.887196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=30 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:20:08.045053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:01.906784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d167d8-14f3-4412-8b0d-fbdcf4b1080b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:01.907276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d167d8-14f3-4412-8b0d-fbdcf4b1080b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:01.907369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d167d8-14f3-4412-8b0d-fbdcf4b1080b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:01.914750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbde76-7eb8-405a-afb0-de870355e7d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:01.915216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbde76-7eb8-405a-afb0-de870355e7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:01.915302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbde76-7eb8-405a-afb0-de870355e7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:01.918436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 657/1307] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:01.998975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d9dbde76-7eb8-405a-afb0-de870355e7d4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:02.000354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d9dbde76-7eb8-405a-afb0-de870355e7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:02.140156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 651/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:21:02.430584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d647c-bb49-4122-b717-7d30cd997caa None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:02.430976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d647c-bb49-4122-b717-7d30cd997caa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:02.431316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d647c-bb49-4122-b717-7d30cd997caa None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:02.432790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 658/1309] 10.222.0.25 () {54 vars in 965 bytes} [Fri Sep 9 06:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:02.439009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a5c62-00d6-4e4f-868e-7122a53a14e3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:02.439298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a5c62-00d6-4e4f-868e-7122a53a14e3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:02.439597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a5c62-00d6-4e4f-868e-7122a53a14e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:02.529062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fe4a5c62-00d6-4e4f-868e-7122a53a14e3 None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:02.530398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe4a5c62-00d6-4e4f-868e-7122a53a14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:02.558978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:02.624555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 652/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Fri Sep 9 06:21:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:02.634429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d647c-bb49-4122-b717-7d30cd997caa None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:02.636203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fd8d647c-bb49-4122-b717-7d30cd997caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:02.757249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d647c-bb49-4122-b717-7d30cd997caa None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:02.842943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:02.843583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:02.843872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:02.844142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:02.845855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:02.909813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:02.975204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:02.976836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7518bb4-4694-4e55-b532-d64341afa63e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:02.999677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 659/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:05.464723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b1d23-ffe4-43eb-8976-596a7927cf7e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:05.465280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b1d23-ffe4-43eb-8976-596a7927cf7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:05.465280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b1d23-ffe4-43eb-8976-596a7927cf7e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:05.467222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 653/1312] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:05.471926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a3ad8-2e8e-4fea-8276-2da69f245b02 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:05.472358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a3ad8-2e8e-4fea-8276-2da69f245b02 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:05.472483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a3ad8-2e8e-4fea-8276-2da69f245b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:05.560938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-170a3ad8-2e8e-4fea-8276-2da69f245b02 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:05.562169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-170a3ad8-2e8e-4fea-8276-2da69f245b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:05.670659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 660/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:05.962257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b1d23-ffe4-43eb-8976-596a7927cf7e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:05.963943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-626b1d23-ffe4-43eb-8976-596a7927cf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:06.072980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b1d23-ffe4-43eb-8976-596a7927cf7e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:06.144388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:06.149034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:06.150562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:06.150987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:06.151656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:06.154638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:06.270310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:06.271747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513be86-fb0a-48ab-8b20-4158dfbcb2ac service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:06.289201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 654/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:08.743890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afa85b-8529-4112-a47e-f45845f6a4eb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:08.744452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afa85b-8529-4112-a47e-f45845f6a4eb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:08.744452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afa85b-8529-4112-a47e-f45845f6a4eb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:08.745767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 661/1315] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:08.750880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace705-1051-4799-b997-0a7140cecc79 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:08.751268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace705-1051-4799-b997-0a7140cecc79 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:08.751442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace705-1051-4799-b997-0a7140cecc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:08.816727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ccace705-1051-4799-b997-0a7140cecc79 None None] MFA Rules not processed for user `af83574daa4c4563a4f76e72696e04f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:08.817859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ccace705-1051-4799-b997-0a7140cecc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:08.869613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:08.914065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 655/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:09.199082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afa85b-8529-4112-a47e-f45845f6a4eb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:09.200668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e4afa85b-8529-4112-a47e-f45845f6a4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:09.210526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:09.303938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afa85b-8529-4112-a47e-f45845f6a4eb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:09.377026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:09.377626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:09.377902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:09.378172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:09.379818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:09.441858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:09.489590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:09.491025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49188d9e-e381-46e1-9be0-ebd66dec59ce service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:09.509502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 662/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:12.190061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae15f4-d421-4c03-b0ce-29214b790937 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:12.190605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae15f4-d421-4c03-b0ce-29214b790937 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:12.190605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae15f4-d421-4c03-b0ce-29214b790937 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:12.192049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 656/1318] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:12.197431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965081d-a84b-4ee2-b53f-a1e533672147 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:12.197847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965081d-a84b-4ee2-b53f-a1e533672147 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:12.197965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965081d-a84b-4ee2-b53f-a1e533672147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:12.262660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1965081d-a84b-4ee2-b53f-a1e533672147 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:12.263885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1965081d-a84b-4ee2-b53f-a1e533672147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:12.358665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 663/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:21:12.671851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7d9ea-4dde-41b7-8b1e-1968a0ba9e1c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:12.672134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7d9ea-4dde-41b7-8b1e-1968a0ba9e1c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:12.672409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7d9ea-4dde-41b7-8b1e-1968a0ba9e1c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:12.673747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 657/1320] 10.222.0.25 () {54 vars in 965 bytes} [Fri Sep 9 06:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:12.680233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6c53-2eac-4f8e-8352-40ea950d9563 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:12.680514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6c53-2eac-4f8e-8352-40ea950d9563 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:12.680779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6c53-2eac-4f8e-8352-40ea950d9563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:12.702451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:12.755989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7fea6c53-2eac-4f8e-8352-40ea950d9563 None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:12.757205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7fea6c53-2eac-4f8e-8352-40ea950d9563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:12.830495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 664/1321] 10.222.0.25 () {58 vars in 1062 bytes} [Fri Sep 9 06:21:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:12.839845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7d9ea-4dde-41b7-8b1e-1968a0ba9e1c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:12.841369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20a7d9ea-4dde-41b7-8b1e-1968a0ba9e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:12.945925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7d9ea-4dde-41b7-8b1e-1968a0ba9e1c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:13.020933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:13.021551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:13.021829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:13.022103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:13.023785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:13.030793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:13.132127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:13.133473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa5a1c-3005-4fd9-8a42-c1606615d052 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:13.151423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 658/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:16.387178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a297-48d3-430e-b3df-6fb07a16bceb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:16.387763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a297-48d3-430e-b3df-6fb07a16bceb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:16.387763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a297-48d3-430e-b3df-6fb07a16bceb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:16.389175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 665/1323] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:16.394473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda99191-e37d-45c9-8be7-bbbed023b59a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:16.394884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda99191-e37d-45c9-8be7-bbbed023b59a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:16.395020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda99191-e37d-45c9-8be7-bbbed023b59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:16.465626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eda99191-e37d-45c9-8be7-bbbed023b59a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:16.466854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eda99191-e37d-45c9-8be7-bbbed023b59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:16.530670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:16.576225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 659/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:16.861259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a297-48d3-430e-b3df-6fb07a16bceb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:16.862775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-caa1a297-48d3-430e-b3df-6fb07a16bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:16.970938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a297-48d3-430e-b3df-6fb07a16bceb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:16.983786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:17.045774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:17.046383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:17.046660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:17.046930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:17.048551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:17.159802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:17.161288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddd494-fc62-43ea-b31f-4389776084be service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:17.180980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 666/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:19.383211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eeb46-8328-4b98-98a4-fbcee413a8ec None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:19.383810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eeb46-8328-4b98-98a4-fbcee413a8ec None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:19.383810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eeb46-8328-4b98-98a4-fbcee413a8ec None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:19.385131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 660/1326] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:19.390478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73fa3-7ebf-47f7-b46e-ffeef45d43e9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:19.390921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73fa3-7ebf-47f7-b46e-ffeef45d43e9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:19.391099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73fa3-7ebf-47f7-b46e-ffeef45d43e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:19.438862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:19.458103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2fc73fa3-7ebf-47f7-b46e-ffeef45d43e9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:19.459998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2fc73fa3-7ebf-47f7-b46e-ffeef45d43e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:19.562465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 667/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:19.842870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583eeb46-8328-4b98-98a4-fbcee413a8ec None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:19.844483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-583eeb46-8328-4b98-98a4-fbcee413a8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:19.950823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583eeb46-8328-4b98-98a4-fbcee413a8ec None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:20.026903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:20.027506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:20.027809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:20.028078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:20.029642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:20.047908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:20.142227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:20.143594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7183bc0-1008-4610-9f68-d07a36a3cfb6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:20.163268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 661/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:21:22.405911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7a16d-5cb6-498c-ace1-a3ccee178037 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:22.406432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7a16d-5cb6-498c-ace1-a3ccee178037 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:22.406547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7a16d-5cb6-498c-ace1-a3ccee178037 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:22.407861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 668/1329] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:22.413094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde17b4-0a49-4edd-b538-4b1435af8507 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:22.413502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde17b4-0a49-4edd-b538-4b1435af8507 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:22.413639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde17b4-0a49-4edd-b538-4b1435af8507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:22.480558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6fde17b4-0a49-4edd-b538-4b1435af8507 None None] MFA Rules not processed for user `af83574daa4c4563a4f76e72696e04f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:22.481696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6fde17b4-0a49-4edd-b538-4b1435af8507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:22.548009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:22.581123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 662/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:22.870035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7a16d-5cb6-498c-ace1-a3ccee178037 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:22.871597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bd7a16d-5cb6-498c-ace1-a3ccee178037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:22.983718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7a16d-5cb6-498c-ace1-a3ccee178037 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:23.005155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:23.060979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:23.061501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:23.061775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:23.062041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:23.063638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:23.180369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:23.181828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c39acd-1191-4deb-b2ad-f4c6cdb658b4 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:23.202275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 669/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:25.617687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2a36a-5ae1-42a9-837a-0c8141f531c3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:25.618222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2a36a-5ae1-42a9-837a-0c8141f531c3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:25.618222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2a36a-5ae1-42a9-837a-0c8141f531c3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:25.619602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 663/1332] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:25.624786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad9de-321e-4fdf-889a-af5168789e60 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:25.625307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad9de-321e-4fdf-889a-af5168789e60 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:25.625416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad9de-321e-4fdf-889a-af5168789e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:25.688497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:25.691405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ed4ad9de-321e-4fdf-889a-af5168789e60 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:25.693263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ed4ad9de-321e-4fdf-889a-af5168789e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:25.787943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 670/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:26.075612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2a36a-5ae1-42a9-837a-0c8141f531c3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:26.077144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b5f2a36a-5ae1-42a9-837a-0c8141f531c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:26.180814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2a36a-5ae1-42a9-837a-0c8141f531c3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:26.253423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:26.254032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:26.254308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:26.254578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:26.256312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:26.283245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:26.367455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:26.368840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de941a1b-a931-498d-8ff2-79c284b90c75 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:26.386509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 664/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:28.594112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f809c-3675-4569-8a52-09544261a72a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:28.594621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f809c-3675-4569-8a52-09544261a72a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:28.594748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f809c-3675-4569-8a52-09544261a72a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:28.596054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 671/1335] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:28.601484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273e516-f3db-4f1a-ba52-456a2af0cb2f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:28.601901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273e516-f3db-4f1a-ba52-456a2af0cb2f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:28.602035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273e516-f3db-4f1a-ba52-456a2af0cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:28.667870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f273e516-f3db-4f1a-ba52-456a2af0cb2f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:28.669080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f273e516-f3db-4f1a-ba52-456a2af0cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:28.743282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:28.766016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 665/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:29.054555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f809c-3675-4569-8a52-09544261a72a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:29.056108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-441f809c-3675-4569-8a52-09544261a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:29.158498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f809c-3675-4569-8a52-09544261a72a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:29.187592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:29.233109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:29.233703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:29.233978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:29.234246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:29.235837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:29.344227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:29.345608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34afa16-5753-4221-876a-e2dc7af56aa0 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:29.363725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 672/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:31.843134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c32f0b-421d-42be-89f1-2f915398e0b4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:31.843716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c32f0b-421d-42be-89f1-2f915398e0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:31.843716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c32f0b-421d-42be-89f1-2f915398e0b4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:31.844985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 666/1338] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:31.850451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072f1b-b604-4b7f-af48-33ab88cebe12 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:31.850925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072f1b-b604-4b7f-af48-33ab88cebe12 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:31.850925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad072f1b-b604-4b7f-af48-33ab88cebe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:31.915425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ad072f1b-b604-4b7f-af48-33ab88cebe12 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:31.916663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ad072f1b-b604-4b7f-af48-33ab88cebe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:31.924844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:32.017315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 673/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:32.306438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c32f0b-421d-42be-89f1-2f915398e0b4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:32.308020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5c32f0b-421d-42be-89f1-2f915398e0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:32.416893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c32f0b-421d-42be-89f1-2f915398e0b4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:32.489244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:32.489771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:32.490044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:32.490308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:32.492754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:32.529381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:32.607263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:32.608729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735d1d8-1df3-4f1b-a961-6dbd412d6ddf service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:32.626908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 667/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:36.764573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432c621-0117-4f71-8a20-33fbf0b1704d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:36.765182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432c621-0117-4f71-8a20-33fbf0b1704d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:36.765182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432c621-0117-4f71-8a20-33fbf0b1704d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:36.766416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 674/1341] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:36.771718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330719-dd2f-4e69-891f-73451e5d550f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:36.772140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330719-dd2f-4e69-891f-73451e5d550f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:36.772406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330719-dd2f-4e69-891f-73451e5d550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:36.839238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e4330719-dd2f-4e69-891f-73451e5d550f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:36.840534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4330719-dd2f-4e69-891f-73451e5d550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:36.926887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:36.939245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 668/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:37.226198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432c621-0117-4f71-8a20-33fbf0b1704d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:37.227754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a432c621-0117-4f71-8a20-33fbf0b1704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:37.332578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432c621-0117-4f71-8a20-33fbf0b1704d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:37.370600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:37.407365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:37.407924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:37.408202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:37.408471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:37.410051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:37.520800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:37.522275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eebd97-9611-43a3-b8ec-bc5fd65729a6 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:37.541901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 675/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:40.384870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865b2fc-1f71-425d-9f87-74ed5053b42f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:40.385438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865b2fc-1f71-425d-9f87-74ed5053b42f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:40.385438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865b2fc-1f71-425d-9f87-74ed5053b42f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:40.386781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 669/1344] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:21:40.392015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33efcc-508a-4c5d-908c-6f5b74eadafc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:40.392435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33efcc-508a-4c5d-908c-6f5b74eadafc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:40.392557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33efcc-508a-4c5d-908c-6f5b74eadafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:40.460500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9a33efcc-508a-4c5d-908c-6f5b74eadafc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:40.461687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9a33efcc-508a-4c5d-908c-6f5b74eadafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:40.481009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:40.564887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 676/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:40.858953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865b2fc-1f71-425d-9f87-74ed5053b42f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:40.860523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d865b2fc-1f71-425d-9f87-74ed5053b42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:40.973538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865b2fc-1f71-425d-9f87-74ed5053b42f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:41.047639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:41.048175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:41.048458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:41.048734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:41.050327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:41.094330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:41.164464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:41.165944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f59c8-c58e-4229-a74d-e9848d15ee74 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:41.184493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 670/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:45.330673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025707c-fad0-4a19-9668-3b2b7d568569 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:45.331173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025707c-fad0-4a19-9668-3b2b7d568569 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:45.331265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025707c-fad0-4a19-9668-3b2b7d568569 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:45.332599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 677/1347] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:45.338279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e9c12-94ff-4360-8209-1af15fe4b4c4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:45.338705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e9c12-94ff-4360-8209-1af15fe4b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:45.338885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e9c12-94ff-4360-8209-1af15fe4b4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:45.412124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-253e9c12-94ff-4360-8209-1af15fe4b4c4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:45.413466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-253e9c12-94ff-4360-8209-1af15fe4b4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:45.523841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:45.527706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 671/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:45.810327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025707c-fad0-4a19-9668-3b2b7d568569 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:21:45.811862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b025707c-fad0-4a19-9668-3b2b7d568569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:45.914932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025707c-fad0-4a19-9668-3b2b7d568569 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:21:45.964116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:45.989760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:21:45.990361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:45.990640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:45.990909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:45.992632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:46.101598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:21:46.102967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e9ca7-c1e8-4d10-bd5b-622406eda82b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:21:46.121643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 678/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:21:51.125504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e09d70-c1c1-4b79-9594-c2a305121bce None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:51.126251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e09d70-c1c1-4b79-9594-c2a305121bce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:51.126696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e09d70-c1c1-4b79-9594-c2a305121bce None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:51.128622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 672/1350] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:51.134611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad82a5-9b24-485a-b62c-3cffa103ad27 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:51.135003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad82a5-9b24-485a-b62c-3cffa103ad27 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:51.135003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad82a5-9b24-485a-b62c-3cffa103ad27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:51.240746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e8ad82a5-9b24-485a-b62c-3cffa103ad27 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:51.242486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e8ad82a5-9b24-485a-b62c-3cffa103ad27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:51.287440 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:51.400462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 679/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:21:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:55.662302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74504210-9c91-4ba1-a72f-72ad025785f9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:55.662870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74504210-9c91-4ba1-a72f-72ad025785f9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:55.663261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74504210-9c91-4ba1-a72f-72ad025785f9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:55.665188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 673/1352] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:55.670970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e8ae-3d1a-46a7-b807-e2bd43b80057 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:55.671421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e8ae-3d1a-46a7-b807-e2bd43b80057 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:55.671774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e8ae-3d1a-46a7-b807-e2bd43b80057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:55.752357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5690e8ae-3d1a-46a7-b807-e2bd43b80057 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:55.753723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5690e8ae-3d1a-46a7-b807-e2bd43b80057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:55.829336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:21:55.872339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 680/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:21:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:21:57.115038 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45c44-cd2b-44d9-bc22-d13d5acf0f03 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:57.115620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45c44-cd2b-44d9-bc22-d13d5acf0f03 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:57.115620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45c44-cd2b-44d9-bc22-d13d5acf0f03 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:57.117156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 674/1354] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:21:57.122348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91297192-7327-46e1-8540-f70dd26aa4b5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:21:57.122808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91297192-7327-46e1-8540-f70dd26aa4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:21:57.122915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91297192-7327-46e1-8540-f70dd26aa4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:21:57.203399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-91297192-7327-46e1-8540-f70dd26aa4b5 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:21:57.204649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91297192-7327-46e1-8540-f70dd26aa4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:21:57.313565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 681/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:21:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:00.966793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d3c0-9031-4811-ae68-73d776a6f35a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:00.967364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d3c0-9031-4811-ae68-73d776a6f35a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:00.967364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d3c0-9031-4811-ae68-73d776a6f35a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:00.968878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 675/1356] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:00.974809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27830-3ab8-44b8-8f7e-ae606d32e3a8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:00.975220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27830-3ab8-44b8-8f7e-ae606d32e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:00.984229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27830-3ab8-44b8-8f7e-ae606d32e3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:01.024170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:01.058092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-14e27830-3ab8-44b8-8f7e-ae606d32e3a8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:01.059296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-14e27830-3ab8-44b8-8f7e-ae606d32e3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:01.171691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 682/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:01.469777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19e85-c3fc-4fc8-b0f5-25d7251caf42 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:01.470312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19e85-c3fc-4fc8-b0f5-25d7251caf42 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:01.470846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19e85-c3fc-4fc8-b0f5-25d7251caf42 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:01.472926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 676/1358] 10.222.0.25 () {54 vars in 962 bytes} [Fri Sep 9 06:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:01.489304 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5fb95-81c1-40c1-8447-6d7c292902d9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:01.490081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5fb95-81c1-40c1-8447-6d7c292902d9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:01.491046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5fb95-81c1-40c1-8447-6d7c292902d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:01.579055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-efa5fb95-81c1-40c1-8447-6d7c292902d9 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:01.580413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-efa5fb95-81c1-40c1-8447-6d7c292902d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:01.645977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:01.730195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 683/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Fri Sep 9 06:22:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:01.737915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19e85-c3fc-4fc8-b0f5-25d7251caf42 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:01.740363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bed19e85-c3fc-4fc8-b0f5-25d7251caf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:01.904929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19e85-c3fc-4fc8-b0f5-25d7251caf42 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:01.986572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:02.034067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:02.034699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:02.034973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:02.035247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:02.036922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:02.137823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:02.139426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ea1ca-12a5-4a29-94b3-6eca58d415a8 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:02.161218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 677/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:22:04.568035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91971862-5ca1-4295-9d89-5b92db4ae5d2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:04.568035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91971862-5ca1-4295-9d89-5b92db4ae5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:04.568035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91971862-5ca1-4295-9d89-5b92db4ae5d2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:04.576443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 684/1361] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:04.581775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1333e-5119-4d54-8259-24255e5f1d7f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:04.582676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1333e-5119-4d54-8259-24255e5f1d7f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:04.582676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1333e-5119-4d54-8259-24255e5f1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:04.661676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1ee1333e-5119-4d54-8259-24255e5f1d7f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:04.663212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1ee1333e-5119-4d54-8259-24255e5f1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:04.705270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:04.769724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 678/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:05.073844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91971862-5ca1-4295-9d89-5b92db4ae5d2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:05.075342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91971862-5ca1-4295-9d89-5b92db4ae5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:05.261165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91971862-5ca1-4295-9d89-5b92db4ae5d2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:05.272632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:05.404517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:05.405167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:05.405447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:05.405706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:05.407298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:05.503667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:05.505278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518aef9-1892-455e-9a62-fd0cb02d7178 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:05.524574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 685/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:08.078383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80434-2386-4549-8953-044bb72e232a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:08.078963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80434-2386-4549-8953-044bb72e232a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:08.078963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80434-2386-4549-8953-044bb72e232a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:08.080450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 679/1364] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:08.085446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a5f3-f467-48e2-8791-9fb4d6f039ac None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:08.085855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a5f3-f467-48e2-8791-9fb4d6f039ac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:08.086052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8a5f3-f467-48e2-8791-9fb4d6f039ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:08.103153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:08.158579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-06c8a5f3-f467-48e2-8791-9fb4d6f039ac None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:08.159995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-06c8a5f3-f467-48e2-8791-9fb4d6f039ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:08.267113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 686/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:08.561503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044f8cc-33d7-47d2-a984-ecce9cb75425 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:08.561991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044f8cc-33d7-47d2-a984-ecce9cb75425 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:08.562268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044f8cc-33d7-47d2-a984-ecce9cb75425 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:08.564173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 680/1366] 10.222.0.25 () {54 vars in 962 bytes} [Fri Sep 9 06:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:08.570814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a185395-5bbb-4bfc-8c4c-ad3035576103 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:08.571097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a185395-5bbb-4bfc-8c4c-ad3035576103 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:08.571402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a185395-5bbb-4bfc-8c4c-ad3035576103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:08.651570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6a185395-5bbb-4bfc-8c4c-ad3035576103 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:08.652820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6a185395-5bbb-4bfc-8c4c-ad3035576103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:08.691199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:08.785035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 687/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Fri Sep 9 06:22:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:08.794002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044f8cc-33d7-47d2-a984-ecce9cb75425 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:08.795723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8044f8cc-33d7-47d2-a984-ecce9cb75425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:08.953876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044f8cc-33d7-47d2-a984-ecce9cb75425 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:08.974818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:09.084217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:09.084868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:09.085132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:09.085387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:09.087799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:09.191891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:09.193626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667ce46-1b56-4559-abb3-84537025ffdf service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:09.214460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 681/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:22:09.229305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9cca-9bc4-45b6-881e-ce6b22e64ba7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:09.229802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9cca-9bc4-45b6-881e-ce6b22e64ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:09.229914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9cca-9bc4-45b6-881e-ce6b22e64ba7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:09.231338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 688/1369] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:09.236380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf9dce-5523-4e7f-9aaf-d46554760234 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:09.236665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf9dce-5523-4e7f-9aaf-d46554760234 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:09.236933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf9dce-5523-4e7f-9aaf-d46554760234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:09.283963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:09.317008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7ccf9dce-5523-4e7f-9aaf-d46554760234 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:09.318336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7ccf9dce-5523-4e7f-9aaf-d46554760234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:09.455200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 682/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:09.754189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ea0c-ae2f-4aa2-b5fe-b3af487b1a60 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:09.754473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ea0c-ae2f-4aa2-b5fe-b3af487b1a60 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:09.754748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ea0c-ae2f-4aa2-b5fe-b3af487b1a60 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:09.756294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 689/1371] 10.222.0.25 () {54 vars in 964 bytes} [Fri Sep 9 06:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:09.761055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fe16e-4162-43d4-a49a-57983b913a43 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:09.761341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fe16e-4162-43d4-a49a-57983b913a43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:09.761614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fe16e-4162-43d4-a49a-57983b913a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:09.842712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f27fe16e-4162-43d4-a49a-57983b913a43 None None] MFA Rules not processed for user `f236664fa09b4a4aafe6980768fc263b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:09.843862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f27fe16e-4162-43d4-a49a-57983b913a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:09.898148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:09.931714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 683/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Fri Sep 9 06:22:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:09.938462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ea0c-ae2f-4aa2-b5fe-b3af487b1a60 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:09.940337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bad6ea0c-ae2f-4aa2-b5fe-b3af487b1a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:10.060492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ea0c-ae2f-4aa2-b5fe-b3af487b1a60 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:10.107849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:10.149199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:10.150091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:10.150640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:10.151069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:10.152933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:10.313335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:10.314952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859d372-8bc5-4d6f-8d69-0c3ae42235b2 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:10.348173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 690/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:11.480834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420be6-c2b3-426c-9778-6ef59b0a197b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:11.481378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420be6-c2b3-426c-9778-6ef59b0a197b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:11.481378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420be6-c2b3-426c-9778-6ef59b0a197b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:11.482761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 684/1374] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:11.488117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226cb8-4204-4c68-9bad-d0e1c3fb78b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:11.488532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226cb8-4204-4c68-9bad-d0e1c3fb78b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:11.488816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226cb8-4204-4c68-9bad-d0e1c3fb78b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:11.556267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-56226cb8-4204-4c68-9bad-d0e1c3fb78b0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:11.557492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-56226cb8-4204-4c68-9bad-d0e1c3fb78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:11.579297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:11.657518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 691/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:11.953488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420be6-c2b3-426c-9778-6ef59b0a197b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:11.955024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bf420be6-c2b3-426c-9778-6ef59b0a197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:12.110764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420be6-c2b3-426c-9778-6ef59b0a197b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:12.165452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:12.232033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:12.233202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:12.233202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:12.233202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:12.234716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:12.332278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:12.333969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48880962-f54f-4aae-967a-7ed8a7c3da91 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:12.352487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 685/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:14.592441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ac0a-1364-4293-94df-e28066a93e61 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:14.592958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ac0a-1364-4293-94df-e28066a93e61 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:14.593039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ac0a-1364-4293-94df-e28066a93e61 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:14.594468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 692/1377] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:14.599777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ebc75-61ca-46ef-a50a-d2fb8fabae33 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:14.600238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ebc75-61ca-46ef-a50a-d2fb8fabae33 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:14.600319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ebc75-61ca-46ef-a50a-d2fb8fabae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:14.675283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-932ebc75-61ca-46ef-a50a-d2fb8fabae33 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:14.676655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-932ebc75-61ca-46ef-a50a-d2fb8fabae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:14.685193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:14.781556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 686/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:15.079615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ac0a-1364-4293-94df-e28066a93e61 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:15.081364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2b67ac0a-1364-4293-94df-e28066a93e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:15.239405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ac0a-1364-4293-94df-e28066a93e61 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:15.244862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:15.362081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:15.362609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:15.362940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:15.363220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:15.364821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:15.465506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:15.467107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ffc2c-39f4-416d-bf24-5a0067f1b9fb service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:15.487890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 693/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:17.834894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832614e6-dc11-49af-a89f-95ae4d39e330 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:17.835478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832614e6-dc11-49af-a89f-95ae4d39e330 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:17.835478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832614e6-dc11-49af-a89f-95ae4d39e330 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:17.838201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 687/1380] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:17.845769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df38e53-8274-4013-9e4b-e3678bb7ca36 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:17.846260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df38e53-8274-4013-9e4b-e3678bb7ca36 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:17.846260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df38e53-8274-4013-9e4b-e3678bb7ca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:17.869232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:17.965660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5df38e53-8274-4013-9e4b-e3678bb7ca36 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:17.967158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5df38e53-8274-4013-9e4b-e3678bb7ca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:18.079247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 694/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:18.368228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832614e6-dc11-49af-a89f-95ae4d39e330 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:18.369878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-832614e6-dc11-49af-a89f-95ae4d39e330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:18.510340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:18.522416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832614e6-dc11-49af-a89f-95ae4d39e330 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:18.651068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:18.651705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:18.651956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:18.652273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:18.654199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:18.752968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:18.754603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d6093-b362-4dd3-a3f0-c253fb719ded service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:18.764837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:18.773420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 688/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:21.260123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52076aa0-6c7e-44b8-9346-b4b16ebac83d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:21.260637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52076aa0-6c7e-44b8-9346-b4b16ebac83d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:21.260786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52076aa0-6c7e-44b8-9346-b4b16ebac83d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:21.262409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 695/1383] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:21.268198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5a794-e1ea-442f-8e5c-d756530790c7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:21.268775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5a794-e1ea-442f-8e5c-d756530790c7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:21.268775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5a794-e1ea-442f-8e5c-d756530790c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:21.349051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00d5a794-e1ea-442f-8e5c-d756530790c7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:21.350463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00d5a794-e1ea-442f-8e5c-d756530790c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:21.461268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 689/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:21.780645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52076aa0-6c7e-44b8-9346-b4b16ebac83d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:21.782466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-52076aa0-6c7e-44b8-9346-b4b16ebac83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:21.876468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:21.947832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52076aa0-6c7e-44b8-9346-b4b16ebac83d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:22.065811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:22.066339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:22.066618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:22.066890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:22.068531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:22.123869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:22.164738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:22.166843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff920e8-5565-42d0-8189-ed4ea3783f14 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:22.184885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 696/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:24.398156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081671-7a3e-4ef2-9596-ca1e2075217c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:24.398680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081671-7a3e-4ef2-9596-ca1e2075217c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:24.398680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081671-7a3e-4ef2-9596-ca1e2075217c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:24.401033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 690/1386] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:24.406459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aac7a-fd92-487f-a414-1eff85a9e5df None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:24.406948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aac7a-fd92-487f-a414-1eff85a9e5df None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:24.407043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aac7a-fd92-487f-a414-1eff85a9e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:24.474775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6e0aac7a-fd92-487f-a414-1eff85a9e5df None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:24.476052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6e0aac7a-fd92-487f-a414-1eff85a9e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:24.593487 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 697/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:24.890433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081671-7a3e-4ef2-9596-ca1e2075217c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:24.892021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc081671-7a3e-4ef2-9596-ca1e2075217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:24.989496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:25.075365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081671-7a3e-4ef2-9596-ca1e2075217c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:25.220940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:25.221766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:25.222136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:25.222554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:25.224904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:25.269878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:25.331373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:25.333326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92601e22-3f37-430f-beb3-86da60779777 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:25.353639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 691/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:27.569151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4dc0d-69eb-410e-8609-faa67af45a46 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:27.572328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4dc0d-69eb-410e-8609-faa67af45a46 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:27.572613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4dc0d-69eb-410e-8609-faa67af45a46 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:27.574137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 698/1389] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:27.579891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de6352-85eb-447b-9c7e-bc391131d9ad None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:27.580752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de6352-85eb-447b-9c7e-bc391131d9ad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:27.581046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de6352-85eb-447b-9c7e-bc391131d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:27.654381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-15de6352-85eb-447b-9c7e-bc391131d9ad None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:27.655623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-15de6352-85eb-447b-9c7e-bc391131d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:27.753961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 692/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:28.049932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4dc0d-69eb-410e-8609-faa67af45a46 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:28.051428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a4d4dc0d-69eb-410e-8609-faa67af45a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:28.084376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:28.203755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4dc0d-69eb-410e-8609-faa67af45a46 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:28.307746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:28.319565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:28.320189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:28.320464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:28.320730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:28.322929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:28.413524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:28.416258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ce55f-e41e-46ca-a1a1-909c245dacb4 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:28.434809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 699/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:28.672467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff038ad-3fb9-449e-bb90-02b98ab3786e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:28.673158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff038ad-3fb9-449e-bb90-02b98ab3786e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:28.674184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff038ad-3fb9-449e-bb90-02b98ab3786e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:28.675601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 693/1392] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:28.682101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b892b34-ed30-4a6a-9e61-5f576f5f75e0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:28.682631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b892b34-ed30-4a6a-9e61-5f576f5f75e0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:28.682631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b892b34-ed30-4a6a-9e61-5f576f5f75e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:28.749769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1b892b34-ed30-4a6a-9e61-5f576f5f75e0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:28.751049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b892b34-ed30-4a6a-9e61-5f576f5f75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:28.811862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:28.851071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 700/1393] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:29.152755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db17ed8-9536-4fde-897e-e2a36f352976 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:29.153049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db17ed8-9536-4fde-897e-e2a36f352976 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:29.153359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db17ed8-9536-4fde-897e-e2a36f352976 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:29.154981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 694/1394] 10.222.0.25 () {54 vars in 966 bytes} [Fri Sep 9 06:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:29.160467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16bd23-4b97-4886-aa0b-26d046b8940c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:29.160970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16bd23-4b97-4886-aa0b-26d046b8940c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:29.161248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16bd23-4b97-4886-aa0b-26d046b8940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:29.240674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fe16bd23-4b97-4886-aa0b-26d046b8940c None None] MFA Rules not processed for user `0f8bf35f310b4b558dde597cde49655f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:29.241907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fe16bd23-4b97-4886-aa0b-26d046b8940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:29.317890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 701/1395] 10.222.0.25 () {58 vars in 1063 bytes} [Fri Sep 9 06:22:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:29.327761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db17ed8-9536-4fde-897e-e2a36f352976 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:29.329261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0db17ed8-9536-4fde-897e-e2a36f352976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:29.344237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:29.440263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db17ed8-9536-4fde-897e-e2a36f352976 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:29.513849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:29.514448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:29.514726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:29.514996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:29.516658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:29.585938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:29.627879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:29.629310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816a08c-57b4-4298-92af-b3bd230a6743 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:29.648044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 695/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:30.385442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df9bbe-e445-4dfc-a61d-65c3215c8052 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:30.386021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df9bbe-e445-4dfc-a61d-65c3215c8052 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:30.386280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df9bbe-e445-4dfc-a61d-65c3215c8052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:30.455155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-53df9bbe-e445-4dfc-a61d-65c3215c8052 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:30.456395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-53df9bbe-e445-4dfc-a61d-65c3215c8052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:30.465571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:30.545680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 702/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:22:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:30.594698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50eef9c-d423-4782-9b5c-9ba0bf39fc7d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:30.595216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50eef9c-d423-4782-9b5c-9ba0bf39fc7d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:30.595337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50eef9c-d423-4782-9b5c-9ba0bf39fc7d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:30.596668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 696/1398] 10.222.0.25 () {54 vars in 943 bytes} [Fri Sep 9 06:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:30.603384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:30.603703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:30.603985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:30.676543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699ed8b-c313-4f2f-962d-6b883505cf25 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:30.677632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699ed8b-c313-4f2f-962d-6b883505cf25 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:30.677917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699ed8b-c313-4f2f-962d-6b883505cf25 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:30.679254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 697/1399] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:22:30.684589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:30.684868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:30.685142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:30.685260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] MFA Rules not processed for user `cb474b7f08db4ff9b9fdf8eb8ca658af`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:30.686451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:30.753057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:30.754161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:30.773715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:30.796855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 703/1400] 10.222.0.25 () {58 vars in 1040 bytes} [Fri Sep 9 06:22:30 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:30.806924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:30.817241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef30b0-7c3e-4a05-809a-5f3ed5d074a1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:30.856467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 698/1401] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:30.921888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:30.922493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:30.922771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:30.923045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:30.924669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:31.012728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:31.014248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab5e7-80bb-45ca-b493-b1bd11fbb18c service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:31.022623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 704/1402] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:22:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 06:22:31.154428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:31.155688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:31.184854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:31.289376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95a76-89d6-47fd-8dcf-a5aec95abced None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:31.423564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:31.424340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:31.424717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:31.425088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:31.427180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:31.448497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:31.535786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:31.537381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7982c0-306f-456b-9702-d19790ac49f5 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:31.558998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 699/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:33.778932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9274a5-71d4-41f9-880c-3a07539b4c85 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:33.779534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9274a5-71d4-41f9-880c-3a07539b4c85 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:33.779534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9274a5-71d4-41f9-880c-3a07539b4c85 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:33.786307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 705/1404] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:33.788860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3abf5-ce42-4d2f-97b2-6e07e82c7f57 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:33.788860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3abf5-ce42-4d2f-97b2-6e07e82c7f57 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:33.789412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3abf5-ce42-4d2f-97b2-6e07e82c7f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:33.909135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7df3abf5-ce42-4d2f-97b2-6e07e82c7f57 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:33.910397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7df3abf5-ce42-4d2f-97b2-6e07e82c7f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:34.013373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:34.020166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 700/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:34.315858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9274a5-71d4-41f9-880c-3a07539b4c85 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:34.318923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bd9274a5-71d4-41f9-880c-3a07539b4c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:34.328490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:34.484385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9274a5-71d4-41f9-880c-3a07539b4c85 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:34.591026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:34.631302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:34.632205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:34.632484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:34.632760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:34.635334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:34.738406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:34.740202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78824d3-1073-46fe-8324-c0edda181cf0 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:34.759626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 706/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:36.982790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5194ec4-ac30-4be3-b691-07a86c96dd6e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:36.983373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5194ec4-ac30-4be3-b691-07a86c96dd6e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:36.983373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5194ec4-ac30-4be3-b691-07a86c96dd6e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:36.984874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 701/1407] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:36.990030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ac5e-36a8-408b-a288-55674ba07914 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:36.990460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ac5e-36a8-408b-a288-55674ba07914 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:36.990618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ac5e-36a8-408b-a288-55674ba07914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:37.062770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a274ac5e-36a8-408b-a288-55674ba07914 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:37.064133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a274ac5e-36a8-408b-a288-55674ba07914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:37.120269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:37.179455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 707/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:37.473636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5194ec4-ac30-4be3-b691-07a86c96dd6e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:37.475159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5194ec4-ac30-4be3-b691-07a86c96dd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:37.639311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5194ec4-ac30-4be3-b691-07a86c96dd6e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:37.720455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:37.768639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:37.769659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:37.769932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:37.770198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:37.772003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:37.864651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:37.866276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5e1e0-8d5e-441f-8432-3540f6cf6ffa service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:37.889293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 702/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:47.178114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=23 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:47.886679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=64 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:53.901357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e288b8-7fa3-4219-8091-dee9bb12e353 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:53.901889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e288b8-7fa3-4219-8091-dee9bb12e353 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:53.901889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e288b8-7fa3-4219-8091-dee9bb12e353 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:53.903237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 708/1410] 10.222.0.25 () {54 vars in 918 bytes} [Fri Sep 9 06:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:53.908348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b684b-8f55-4b54-b706-bd46e3decbda None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:53.908766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b684b-8f55-4b54-b706-bd46e3decbda None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:53.908877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b684b-8f55-4b54-b706-bd46e3decbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:53.985115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-386b684b-8f55-4b54-b706-bd46e3decbda None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:53.986305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-386b684b-8f55-4b54-b706-bd46e3decbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:54.089696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 703/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Sep 9 06:22:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:54.104905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f0c4-3b2f-4b6e-a99e-5fa3817689fd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:54.105188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f0c4-3b2f-4b6e-a99e-5fa3817689fd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:54.105466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f0c4-3b2f-4b6e-a99e-5fa3817689fd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:54.106886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 709/1412] 10.222.0.25 () {54 vars in 918 bytes} [Fri Sep 9 06:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:54.111439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688e87e-bfa6-41a9-ae0a-aadf8ec2d963 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:54.111855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688e87e-bfa6-41a9-ae0a-aadf8ec2d963 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:54.112130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688e87e-bfa6-41a9-ae0a-aadf8ec2d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:54.185179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:54.188275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b688e87e-bfa6-41a9-ae0a-aadf8ec2d963 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:54.190071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b688e87e-bfa6-41a9-ae0a-aadf8ec2d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:54.307173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 704/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Sep 9 06:22:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:22:56.896913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4717bf-7b01-4ac9-971c-ba085e251d70 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:56.897531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4717bf-7b01-4ac9-971c-ba085e251d70 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:56.897531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4717bf-7b01-4ac9-971c-ba085e251d70 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:56.898594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:56.904399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586534c-1f00-4643-82c7-de019cc8dc0c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:56.904857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586534c-1f00-4643-82c7-de019cc8dc0c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:56.904964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586534c-1f00-4643-82c7-de019cc8dc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:56.972297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1586534c-1f00-4643-82c7-de019cc8dc0c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:56.973520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1586534c-1f00-4643-82c7-de019cc8dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:57.029407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:57.071985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:57.363566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4717bf-7b01-4ac9-971c-ba085e251d70 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:57.365087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ee4717bf-7b01-4ac9-971c-ba085e251d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:57.630894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4717bf-7b01-4ac9-971c-ba085e251d70 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:57.704216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:57.744731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:57.745333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:57.745609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:57.745880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:57.747452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:57.835527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:57.837082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61559e-45c9-4fb5-a303-cd41df088050 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:57.855381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 711/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:22:59.006859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d03c3-664d-4d0a-a8dc-0dda867625c9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:59.007434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d03c3-664d-4d0a-a8dc-0dda867625c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:59.007434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d03c3-664d-4d0a-a8dc-0dda867625c9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:59.008863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:22:59.015710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93004b-63de-499c-a1f4-d745bc0fbc85 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:59.016157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93004b-63de-499c-a1f4-d745bc0fbc85 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:59.016257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93004b-63de-499c-a1f4-d745bc0fbc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:59.091398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9d93004b-63de-499c-a1f4-d745bc0fbc85 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:22:59.092652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d93004b-63de-499c-a1f4-d745bc0fbc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:59.120042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:59.189041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:22:59.341706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d03c3-664d-4d0a-a8dc-0dda867625c9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:22:59.343133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b7d03c3-664d-4d0a-a8dc-0dda867625c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:59.497106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d03c3-664d-4d0a-a8dc-0dda867625c9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:22:59.536662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:22:59.610874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:22:59.611403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:22:59.611731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:22:59.612005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:22:59.613596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:22:59.706369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:22:59.707986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b043668-8678-4668-b1b8-d6cc7fb9dcaa service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:22:59.726451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 707/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:00.859958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cfc79-2191-48b5-8bac-e4b04c8c1fe8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:00.860561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cfc79-2191-48b5-8bac-e4b04c8c1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:00.860561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cfc79-2191-48b5-8bac-e4b04c8c1fe8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:00.861757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:00.868094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c925ae-3e63-4516-9726-d6ea330a19b7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:00.868511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c925ae-3e63-4516-9726-d6ea330a19b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:00.868640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c925ae-3e63-4516-9726-d6ea330a19b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:00.922006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:00.943665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-86c925ae-3e63-4516-9726-d6ea330a19b7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:00.944876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-86c925ae-3e63-4516-9726-d6ea330a19b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:01.048970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:01.201213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cfc79-2191-48b5-8bac-e4b04c8c1fe8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:01.202726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0a3cfc79-2191-48b5-8bac-e4b04c8c1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:01.347384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cfc79-2191-48b5-8bac-e4b04c8c1fe8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:01.359206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:01.462577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:01.463148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:01.463352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:01.463653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:01.465269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:01.558684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:01.560333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c73a3a-eeb0-445d-a1cb-73f9a02581b7 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:01.579129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 714/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:02.683430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3610d-1558-4266-8ac3-97236aea6644 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:02.684033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3610d-1558-4266-8ac3-97236aea6644 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:02.684033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3610d-1558-4266-8ac3-97236aea6644 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:02.685353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:23:02.691943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64286268-1c0e-4658-af7d-ec1c21740b7c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:02.692364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64286268-1c0e-4658-af7d-ec1c21740b7c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:02.692535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64286268-1c0e-4658-af7d-ec1c21740b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:02.713146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:02.769772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-64286268-1c0e-4658-af7d-ec1c21740b7c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:02.770961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-64286268-1c0e-4658-af7d-ec1c21740b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:02.867123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:23:03.017336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3610d-1558-4266-8ac3-97236aea6644 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:03.018888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7cf3610d-1558-4266-8ac3-97236aea6644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:03.129156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:03.164826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3610d-1558-4266-8ac3-97236aea6644 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:03.278031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:03.278642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:03.278921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:03.279195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:03.280813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:03.358032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:03.372329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:03.373934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d5dba-11ae-45c2-a2cd-61ed75db09fb service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:03.392185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 710/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:04.508669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63640df9-a7be-4d14-b85b-e7d61aa851da None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:04.509239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63640df9-a7be-4d14-b85b-e7d61aa851da None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:04.509239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63640df9-a7be-4d14-b85b-e7d61aa851da None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:04.510670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:04.516799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27815993-7a77-4d51-b7b6-2cc31fa683c8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:04.517261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27815993-7a77-4d51-b7b6-2cc31fa683c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:04.517360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27815993-7a77-4d51-b7b6-2cc31fa683c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:04.593399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-27815993-7a77-4d51-b7b6-2cc31fa683c8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:04.594609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-27815993-7a77-4d51-b7b6-2cc31fa683c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:04.690982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:04.841163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63640df9-a7be-4d14-b85b-e7d61aa851da None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:04.842623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-63640df9-a7be-4d14-b85b-e7d61aa851da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:04.925540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:04.988217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63640df9-a7be-4d14-b85b-e7d61aa851da None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:05.100841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:05.101374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:05.101647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:05.101911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:05.104105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:05.152539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:05.200523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:05.202395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d93a9d-a005-425c-9e1d-6ceaed1670e7 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:05.221018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 717/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:06.282835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951fe67-8770-4490-853b-7a0f936202b1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:06.283411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951fe67-8770-4490-853b-7a0f936202b1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:06.283411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951fe67-8770-4490-853b-7a0f936202b1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:06.284751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:06.290951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f7caa-f121-4c9c-98a8-bd4746a3cb42 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:06.291455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f7caa-f121-4c9c-98a8-bd4746a3cb42 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:06.291830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f7caa-f121-4c9c-98a8-bd4746a3cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:06.366413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d82f7caa-f121-4c9c-98a8-bd4746a3cb42 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:06.367649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d82f7caa-f121-4c9c-98a8-bd4746a3cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:06.465078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:06.617068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951fe67-8770-4490-853b-7a0f936202b1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:06.618605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7951fe67-8770-4490-853b-7a0f936202b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:06.668953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:06.765536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951fe67-8770-4490-853b-7a0f936202b1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:06.880119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:06.880650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:06.880928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:06.881198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:06.882810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:06.890049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:06.981003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:06.982849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d0fb3-cd92-41ec-82ba-4c9184b7294f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:07.002106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 713/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:08.060447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e484-f76d-418d-8807-37611991290c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:08.061027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e484-f76d-418d-8807-37611991290c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:08.061027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e484-f76d-418d-8807-37611991290c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:08.062367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:08.068599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4038d-98d7-4540-adb1-0205ac86d09f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:08.069039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4038d-98d7-4540-adb1-0205ac86d09f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:08.069162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4038d-98d7-4540-adb1-0205ac86d09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:08.143902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-43f4038d-98d7-4540-adb1-0205ac86d09f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:08.145093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43f4038d-98d7-4540-adb1-0205ac86d09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:08.211490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:08.241478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:08.393385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e484-f76d-418d-8807-37611991290c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:08.394872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bcc3e484-f76d-418d-8807-37611991290c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:08.412892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:08.539289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e484-f76d-418d-8807-37611991290c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:08.626195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:08.652032 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:08.652637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:08.652914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:08.653187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:08.654764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:08.742946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:08.745395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee646b9-abf9-4cfd-a64c-f7e3e8400f5c service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:08.763682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 720/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:09.827392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a70e9-23d4-4de8-bd29-077d2c346340 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:09.827950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a70e9-23d4-4de8-bd29-077d2c346340 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:09.827950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a70e9-23d4-4de8-bd29-077d2c346340 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:09.829225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:09.835271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea641aa1-517e-4277-a9ef-c65291e2c53c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:09.835735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea641aa1-517e-4277-a9ef-c65291e2c53c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:09.835830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea641aa1-517e-4277-a9ef-c65291e2c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:09.912732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ea641aa1-517e-4277-a9ef-c65291e2c53c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:09.913924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ea641aa1-517e-4277-a9ef-c65291e2c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:09.954931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:10.009660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:10.161108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a70e9-23d4-4de8-bd29-077d2c346340 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:10.162620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-796a70e9-23d4-4de8-bd29-077d2c346340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:10.305656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a70e9-23d4-4de8-bd29-077d2c346340 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:10.360225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:10.418993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:10.419521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:10.419825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:10.420095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:10.421676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:10.510501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:10.512122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25da719-5799-4e42-9bba-93602855bac6 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:10.529919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 716/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:11.584251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56d132-e443-4176-a129-8bc924e8f812 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:11.584761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56d132-e443-4176-a129-8bc924e8f812 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:11.584858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56d132-e443-4176-a129-8bc924e8f812 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:11.586122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:11.593332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a01f-73c6-4df0-933b-72b745b06f89 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:11.593764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a01f-73c6-4df0-933b-72b745b06f89 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:11.593882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a01f-73c6-4df0-933b-72b745b06f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:11.670404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a7f5a01f-73c6-4df0-933b-72b745b06f89 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:11.671631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a7f5a01f-73c6-4df0-933b-72b745b06f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:11.673651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:11.768289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:11.919624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56d132-e443-4176-a129-8bc924e8f812 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:11.921124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fd56d132-e443-4176-a129-8bc924e8f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:12.065144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56d132-e443-4176-a129-8bc924e8f812 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:12.092806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:12.177894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:12.178418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:12.178694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:12.178962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:12.180561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:12.268789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:12.270380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed14b5-3087-4e76-a169-569fe79372fc service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:12.288691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 723/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:14.339741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559e47c-a6a2-43fd-9315-fc34853b31e5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:14.340325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559e47c-a6a2-43fd-9315-fc34853b31e5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:14.340325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559e47c-a6a2-43fd-9315-fc34853b31e5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:14.341655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:14.347286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86b083-de81-47f1-a1d7-52dc628bc7d7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:14.347723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86b083-de81-47f1-a1d7-52dc628bc7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:14.347874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86b083-de81-47f1-a1d7-52dc628bc7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:14.388482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:14.413872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8c86b083-de81-47f1-a1d7-52dc628bc7d7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:14.415073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8c86b083-de81-47f1-a1d7-52dc628bc7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:14.511521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:14.795221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559e47c-a6a2-43fd-9315-fc34853b31e5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:14.796861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7559e47c-a6a2-43fd-9315-fc34853b31e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:14.899334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559e47c-a6a2-43fd-9315-fc34853b31e5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:14.932611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:14.976944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:14.977546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:14.977823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:14.978093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:14.980374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:15.086625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:15.088049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957bd89-7440-4195-af0e-99530e64b9dc service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:15.106494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 719/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:17.501328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7e60-3be1-4d4c-98b2-ae076de8c2d0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:17.501842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7e60-3be1-4d4c-98b2-ae076de8c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:17.501842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc7e60-3be1-4d4c-98b2-ae076de8c2d0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:17.503177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:17.508384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5608-0366-424d-879c-430d9c02a6a8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:17.508783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5608-0366-424d-879c-430d9c02a6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:17.508936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5608-0366-424d-879c-430d9c02a6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:17.582690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f99b5608-0366-424d-879c-430d9c02a6a8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:17.583936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f99b5608-0366-424d-879c-430d9c02a6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:17.596960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:17.679954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:17.963936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc7e60-3be1-4d4c-98b2-ae076de8c2d0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:17.965463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-15bc7e60-3be1-4d4c-98b2-ae076de8c2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:18.069107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc7e60-3be1-4d4c-98b2-ae076de8c2d0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:18.081794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:18.144338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:18.144959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:18.145236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:18.145555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:18.147120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:18.255187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:18.256551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f7fc8-269c-48a3-b654-2120244318ff service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:18.274047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 726/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:20.722581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e951c5b-f31d-409d-af48-4852d1d19400 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:20.723108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e951c5b-f31d-409d-af48-4852d1d19400 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:20.723108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e951c5b-f31d-409d-af48-4852d1d19400 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:20.724504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:20.729739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942d63b-8558-4171-8256-11c76c5afb42 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:20.730204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942d63b-8558-4171-8256-11c76c5afb42 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:20.730320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942d63b-8558-4171-8256-11c76c5afb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:20.778579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:20.796541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7942d63b-8558-4171-8256-11c76c5afb42 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:20.797829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7942d63b-8558-4171-8256-11c76c5afb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:20.894746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:21.183181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e951c5b-f31d-409d-af48-4852d1d19400 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:21.184753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2e951c5b-f31d-409d-af48-4852d1d19400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:21.290249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e951c5b-f31d-409d-af48-4852d1d19400 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:21.330052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:21.364969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:21.365575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:21.365854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:21.366125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:21.367750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:21.477578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:21.479771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13656c6-3daa-4894-9b56-db1bc2d70d7d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:21.497756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 722/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:23:23.774883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441c5b-7ae0-4971-aea9-5a59a9b75b5e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:23.775736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441c5b-7ae0-4971-aea9-5a59a9b75b5e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:23.775736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441c5b-7ae0-4971-aea9-5a59a9b75b5e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:23.776875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:23.782339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7331c97-75b3-43c5-b666-92f335bc89b9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:23.782774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7331c97-75b3-43c5-b666-92f335bc89b9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:23.782872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7331c97-75b3-43c5-b666-92f335bc89b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:23.847673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b7331c97-75b3-43c5-b666-92f335bc89b9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:23.848787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7331c97-75b3-43c5-b666-92f335bc89b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:23.869671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:23.945781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:24.227672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441c5b-7ae0-4971-aea9-5a59a9b75b5e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:24.229818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-49441c5b-7ae0-4971-aea9-5a59a9b75b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:24.331929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441c5b-7ae0-4971-aea9-5a59a9b75b5e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:24.352072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:24.406338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:24.406944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:24.407224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:24.407497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:24.409113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:24.516754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:24.518108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d222cf-4a66-4bb8-b397-0298f836debb service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:24.536450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 729/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:26.750563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5d2c6-c522-42dd-9fd5-86cfb71fd1f0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:26.751124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5d2c6-c522-42dd-9fd5-86cfb71fd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:26.751124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5d2c6-c522-42dd-9fd5-86cfb71fd1f0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:26.752503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:26.757883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3349feb-001a-462a-86b4-8efebac87d83 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:26.758310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3349feb-001a-462a-86b4-8efebac87d83 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:26.758504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3349feb-001a-462a-86b4-8efebac87d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:26.822524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:26.825452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f3349feb-001a-462a-86b4-8efebac87d83 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:26.827207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3349feb-001a-462a-86b4-8efebac87d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:26.926774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:27.215831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5d2c6-c522-42dd-9fd5-86cfb71fd1f0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:27.216952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-25c5d2c6-c522-42dd-9fd5-86cfb71fd1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:27.321151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5d2c6-c522-42dd-9fd5-86cfb71fd1f0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:27.371347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:27.395748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:27.396350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:27.396627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:27.396897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:27.398476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:27.506008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:27.507441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4fc9d-4303-4330-82a6-e44085b0382a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:27.525797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 725/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:29.753755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3631c9f-b366-4648-ba89-4c9b4bd8925b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:29.754309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3631c9f-b366-4648-ba89-4c9b4bd8925b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:29.754309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3631c9f-b366-4648-ba89-4c9b4bd8925b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:29.755716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:29.761664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3581ca-2f85-44e5-a426-2fadf9756f1a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:29.762103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3581ca-2f85-44e5-a426-2fadf9756f1a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:29.762221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3581ca-2f85-44e5-a426-2fadf9756f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:29.827328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5b3581ca-2f85-44e5-a426-2fadf9756f1a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:29.828504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5b3581ca-2f85-44e5-a426-2fadf9756f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:29.857512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:29.924545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:23:30.207267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3631c9f-b366-4648-ba89-4c9b4bd8925b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:30.208833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d3631c9f-b366-4648-ba89-4c9b4bd8925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:30.312393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3631c9f-b366-4648-ba89-4c9b4bd8925b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:30.344540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:30.389943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:30.390548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:30.390826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:30.391097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:30.392752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:30.500027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:30.501386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e88599-e93a-47ff-877e-14c95d09af02 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:30.518972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 732/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:32.727878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3386e5f-6076-4ea0-9dcd-41f5780d2b72 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:32.728468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3386e5f-6076-4ea0-9dcd-41f5780d2b72 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:32.728468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3386e5f-6076-4ea0-9dcd-41f5780d2b72 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:32.730216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:32.735418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b26cc2-ef32-43e3-a0df-f32c2b4e5213 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:32.735932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b26cc2-ef32-43e3-a0df-f32c2b4e5213 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:32.736049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b26cc2-ef32-43e3-a0df-f32c2b4e5213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:32.800900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-56b26cc2-ef32-43e3-a0df-f32c2b4e5213 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:32.801996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-56b26cc2-ef32-43e3-a0df-f32c2b4e5213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:32.810697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:32.897442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:23:33.180674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3386e5f-6076-4ea0-9dcd-41f5780d2b72 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:33.182341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3386e5f-6076-4ea0-9dcd-41f5780d2b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:33.286748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3386e5f-6076-4ea0-9dcd-41f5780d2b72 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:33.342622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:33.361160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:33.361763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:33.362042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:33.362314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:33.363936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:33.476976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:33.478336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74134458-e4f0-4078-9732-3f41734d565f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:33.497172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 728/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:35.702435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a88ab-18d4-484f-999f-cc011d931e22 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:35.702999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a88ab-18d4-484f-999f-cc011d931e22 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:35.702999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a88ab-18d4-484f-999f-cc011d931e22 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:35.704398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:35.709716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225d1e3-5cc4-4491-8e13-19107bc3cd99 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:35.710112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225d1e3-5cc4-4491-8e13-19107bc3cd99 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:35.710267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225d1e3-5cc4-4491-8e13-19107bc3cd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:35.776027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d225d1e3-5cc4-4491-8e13-19107bc3cd99 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:35.777229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d225d1e3-5cc4-4491-8e13-19107bc3cd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:35.815803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:35.874496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:23:36.158189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a88ab-18d4-484f-999f-cc011d931e22 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:36.159779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c56a88ab-18d4-484f-999f-cc011d931e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:36.264011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a88ab-18d4-484f-999f-cc011d931e22 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:36.302901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:36.339030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:36.339664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:36.339944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:36.340215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:36.341795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:36.449330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:36.450675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f971432-4b7e-48fb-b4d7-4da65eb90a8a service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:36.468816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 735/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:38.648566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c239-b125-44c1-a1d0-7d3d2363cc75 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:38.649133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c239-b125-44c1-a1d0-7d3d2363cc75 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:38.649133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c239-b125-44c1-a1d0-7d3d2363cc75 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:38.650516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:38.656082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb17fe-0e0e-46cb-867b-ad4d4c5c16fc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:38.656522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb17fe-0e0e-46cb-867b-ad4d4c5c16fc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:38.656632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb17fe-0e0e-46cb-867b-ad4d4c5c16fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:38.722220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-30cb17fe-0e0e-46cb-867b-ad4d4c5c16fc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:38.723423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-30cb17fe-0e0e-46cb-867b-ad4d4c5c16fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:38.741053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:38.819001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:39.111200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c239-b125-44c1-a1d0-7d3d2363cc75 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:39.112828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4a1c239-b125-44c1-a1d0-7d3d2363cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:39.215347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c239-b125-44c1-a1d0-7d3d2363cc75 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:39.282864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:39.289452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:39.290049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:39.290323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:39.290590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:39.292830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:39.401294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:39.402771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634cf06-d536-4e2b-8fb4-d187b0f78475 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:39.421537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 731/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:23:41.579439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1887bda-07d5-4fb9-b54d-62aa1649ea50 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:41.580033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1887bda-07d5-4fb9-b54d-62aa1649ea50 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:41.580033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1887bda-07d5-4fb9-b54d-62aa1649ea50 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:41.581366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:23:41.586967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d652-e39a-4a38-a456-510bf582a2ea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:41.587351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d652-e39a-4a38-a456-510bf582a2ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:41.587523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d652-e39a-4a38-a456-510bf582a2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:41.653850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1137d652-e39a-4a38-a456-510bf582a2ea None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:23:41.655065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1137d652-e39a-4a38-a456-510bf582a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:41.705177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:41.758769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:23:42.042814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1887bda-07d5-4fb9-b54d-62aa1649ea50 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:23:42.044378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1887bda-07d5-4fb9-b54d-62aa1649ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:42.148135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1887bda-07d5-4fb9-b54d-62aa1649ea50 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:23:42.199242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:42.226101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:23:42.226701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:23:42.226976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:23:42.227245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:23:42.229085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:23:42.337003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:23:42.338521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06002a-8b2a-4b9f-a5c8-eb487a3310f8 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:23:42.357278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 738/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:23:51.757265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=22 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:23:52.354985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=66 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:02.404798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0569c1-30f6-4b0d-8abf-237059ac1a9f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:02.405413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0569c1-30f6-4b0d-8abf-237059ac1a9f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:02.405413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0569c1-30f6-4b0d-8abf-237059ac1a9f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:02.406701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 733/1471] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:02.412012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7fc2-9bb9-487a-aee5-5f280fcec70d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:02.412492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7fc2-9bb9-487a-aee5-5f280fcec70d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:02.412620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7fc2-9bb9-487a-aee5-5f280fcec70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:02.479721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5fca7fc2-9bb9-487a-aee5-5f280fcec70d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:02.480785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5fca7fc2-9bb9-487a-aee5-5f280fcec70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:02.578183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 739/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:02.888931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc2b39-d082-4898-ac82-f8e213f7e047 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:02.889214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc2b39-d082-4898-ac82-f8e213f7e047 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:02.889488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc2b39-d082-4898-ac82-f8e213f7e047 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:02.890820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 734/1473] 10.222.0.25 () {54 vars in 966 bytes} [Fri Sep 9 06:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:02.896056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51a0b-9266-425b-b879-f18229ffad08 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:02.896341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51a0b-9266-425b-b879-f18229ffad08 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:02.896615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51a0b-9266-425b-b879-f18229ffad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:02.973691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2fc51a0b-9266-425b-b879-f18229ffad08 None None] MFA Rules not processed for user `0f8bf35f310b4b558dde597cde49655f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:02.974907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2fc51a0b-9266-425b-b879-f18229ffad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:02.995088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:03.050623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 740/1474] 10.222.0.25 () {58 vars in 1063 bytes} [Fri Sep 9 06:24:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:03.058656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc2b39-d082-4898-ac82-f8e213f7e047 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:03.060196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c9cc2b39-d082-4898-ac82-f8e213f7e047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:03.163397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc2b39-d082-4898-ac82-f8e213f7e047 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:03.239318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:03.239954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:03.240229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:03.240497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:03.242064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:03.296577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:03.353672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:03.355102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8319b-7ce7-4e8c-af72-dfe39728bd5c service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:03.373907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 735/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:24:05.619633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60385c-3bdd-42a9-84bb-70950505294f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:05.620179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60385c-3bdd-42a9-84bb-70950505294f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:05.620179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60385c-3bdd-42a9-84bb-70950505294f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:05.621468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 741/1476] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:05.626871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e51d-9e5b-48cd-b5fa-2b589e2c2432 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:05.627298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e51d-9e5b-48cd-b5fa-2b589e2c2432 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:05.627405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e51d-9e5b-48cd-b5fa-2b589e2c2432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:05.693019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bbe5e51d-9e5b-48cd-b5fa-2b589e2c2432 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:05.694229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bbe5e51d-9e5b-48cd-b5fa-2b589e2c2432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:05.788965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 736/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:06.072769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60385c-3bdd-42a9-84bb-70950505294f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:06.074290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e60385c-3bdd-42a9-84bb-70950505294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:06.198338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60385c-3bdd-42a9-84bb-70950505294f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:06.247794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:06.293039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:06.293665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:06.293942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:06.294211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:06.294857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:06.296650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:24:06.296650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:24:06.296650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:24:06.297803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc50cc-eba3-4f7c-9d37-29cb2397ba98 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:06.318379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 742/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Sep 9 06:24:06 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:24:08.524491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0bdb-2355-4df9-ab66-3b7ec8161d9c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:08.525086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0bdb-2355-4df9-ab66-3b7ec8161d9c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:08.525086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a0bdb-2355-4df9-ab66-3b7ec8161d9c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:08.527096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 737/1479] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:08.532509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcb4ce-6ef3-4141-944e-9a9b6819f90b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:08.532943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcb4ce-6ef3-4141-944e-9a9b6819f90b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:08.533049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcb4ce-6ef3-4141-944e-9a9b6819f90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:08.600205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dfdcb4ce-6ef3-4141-944e-9a9b6819f90b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:08.601402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dfdcb4ce-6ef3-4141-944e-9a9b6819f90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:08.696394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 743/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:08.992512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a0bdb-2355-4df9-ab66-3b7ec8161d9c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:08.994241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c6a0bdb-2355-4df9-ab66-3b7ec8161d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:09.004020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:09.140098 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a0bdb-2355-4df9-ab66-3b7ec8161d9c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:09.218850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:09.250972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:09.251598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:09.251876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:09.252146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:09.253715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:09.343641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:09.345226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42733c12-7acb-485f-9063-aa0b1bacd86b service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:09.363295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 738/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:24:11.720171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a06a9-3da3-4faa-8794-f0b53f43d592 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:11.720704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a06a9-3da3-4faa-8794-f0b53f43d592 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:11.720704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a06a9-3da3-4faa-8794-f0b53f43d592 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:11.722089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 744/1482] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:11.727636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210c498-2e81-4f26-8a82-e1e05560c0a4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:11.728022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210c498-2e81-4f26-8a82-e1e05560c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:11.728187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210c498-2e81-4f26-8a82-e1e05560c0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:11.794189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7210c498-2e81-4f26-8a82-e1e05560c0a4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:11.795394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7210c498-2e81-4f26-8a82-e1e05560c0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:11.841751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:11.893514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 739/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:12.183907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a06a9-3da3-4faa-8794-f0b53f43d592 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:12.185467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2e1a06a9-3da3-4faa-8794-f0b53f43d592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:12.208986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:12.289558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a06a9-3da3-4faa-8794-f0b53f43d592 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:12.363696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:12.364310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:12.364600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:12.364856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:12.366439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:12.439884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:12.475131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:12.476610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9c965-90c7-40ec-bcc8-12b27c2d670e service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:12.494887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 745/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:24:14.705751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d195-5d88-4b19-8354-7fc9f6852228 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:14.706306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d195-5d88-4b19-8354-7fc9f6852228 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:14.706306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d195-5d88-4b19-8354-7fc9f6852228 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:14.707690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 740/1485] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:14.713173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a1c0b-1e72-460c-8e94-af9f0b148ceb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:14.713732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a1c0b-1e72-460c-8e94-af9f0b148ceb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:14.713732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a1c0b-1e72-460c-8e94-af9f0b148ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:14.778180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a63a1c0b-1e72-460c-8e94-af9f0b148ceb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:14.779369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a63a1c0b-1e72-460c-8e94-af9f0b148ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:14.874209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 746/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:24:15.160949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7d195-5d88-4b19-8354-7fc9f6852228 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:15.162486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9a7d195-5d88-4b19-8354-7fc9f6852228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:15.266051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7d195-5d88-4b19-8354-7fc9f6852228 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:15.342419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:15.344086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:15.344756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:15.345222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:15.345734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:15.347566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:15.457032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:15.458423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53a6d8-733d-4d91-af15-a20e886d25ba service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:15.593656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 741/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:24:17.862409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332163-f2c7-42cd-aa92-7f8b413661d3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:17.862918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332163-f2c7-42cd-aa92-7f8b413661d3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:17.862918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332163-f2c7-42cd-aa92-7f8b413661d3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:17.864287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 747/1488] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:17.870418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b0adc-c481-4511-9898-92da4057009d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:17.870850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b0adc-c481-4511-9898-92da4057009d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:17.870955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b0adc-c481-4511-9898-92da4057009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:17.939173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-641b0adc-c481-4511-9898-92da4057009d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:17.940420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-641b0adc-c481-4511-9898-92da4057009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:17.993693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:18.036930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 742/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:18.329876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332163-f2c7-42cd-aa92-7f8b413661d3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:18.331448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8a332163-f2c7-42cd-aa92-7f8b413661d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:18.367390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:18.435453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332163-f2c7-42cd-aa92-7f8b413661d3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:18.512103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:18.512712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:18.512991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:18.513259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:18.514845 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:18.594086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:18.623849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:18.625299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c998c5-8a48-4a5c-8ee4-358cecbe3756 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:18.643721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 748/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:24:21.021274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edac2e1-e590-4399-9eb9-c26dbda36882 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:21.021817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edac2e1-e590-4399-9eb9-c26dbda36882 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:21.021817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edac2e1-e590-4399-9eb9-c26dbda36882 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:21.023171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 743/1491] 10.222.0.25 () {54 vars in 927 bytes} [Fri Sep 9 06:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:24:21.028834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38a0d8-1a66-44f5-9afa-9460e32418be None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:21.029243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38a0d8-1a66-44f5-9afa-9460e32418be None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:21.029389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38a0d8-1a66-44f5-9afa-9460e32418be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:21.094426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4a38a0d8-1a66-44f5-9afa-9460e32418be None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:24:21.095651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4a38a0d8-1a66-44f5-9afa-9460e32418be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:21.190506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 749/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Fri Sep 9 06:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:24:21.478035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edac2e1-e590-4399-9eb9-c26dbda36882 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:24:21.479718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6edac2e1-e590-4399-9eb9-c26dbda36882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:21.584387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edac2e1-e590-4399-9eb9-c26dbda36882 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:24:21.657555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:24:21.658152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:24:21.658426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:24:21.658691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:24:21.660297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:24:21.670016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:21.769508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:24:21.770842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7f9dd-ca0c-4862-90f5-cdbda1fc79e4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:24:21.789722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 744/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:24:31.189027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=82 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:24:31.787813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=50 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:37.729225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:37.729804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:37.729804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:37.806238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:25:37.807343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:37.902246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 750/1494] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:25:37.910107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:37.910597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:37.910718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:37.985821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:25:37.987026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:38.083596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 745/1495] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:25:38.088756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:38.089962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:38.101484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3cb8d-9fb1-4264-a333-c94c7562e494 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:38.162538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:38.195387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:38.196024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:38.196838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:38.197212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:38.197748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:25:38.198047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:38.199052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:38.199052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:25:38.199052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:25:38.199974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:38.211079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 751/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:25:38.220112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:38.229228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8adea-726a-4ceb-a865-aa6b7485b96f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:38.291949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:38.326678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:38.327285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:38.327583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:38.327859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:38.328487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:38.330214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:38.330214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:25:38.330214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:25:38.331396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:38.388615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 746/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:25:38 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:25:38.392782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:38.394313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:38.428036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638e42b-cfee-41b5-92cb-a818710fddb1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:38.521849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:38.522449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:38.522726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:38.522999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:38.523642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:38.525249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:38.525249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:25:38.525249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:25:38.526437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:38.568962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 752/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:25:38 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:25:38.573567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:38.575080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:38.636243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22491562-aa91-4365-9617-887160a47f2f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:38.719704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:38.731502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:38.732140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:38.732418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:38.732690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:38.733334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:38.734352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:38.734352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:25:38.734352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:25:38.735227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:38.747946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 747/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:25:38.752010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:38.753538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:38.765454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012eb3f-11d5-4e4f-9a46-d79285ce1fa9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:38.790047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:38.861086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:38.861697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:38.861974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:38.862244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:38.862888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:38.863923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:38.863923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:25:38.863923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:25:38.864813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dacf78-1a86-4113-9a04-eaa9d276c7d8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:38.877463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 753/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:25:38.881618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:38.883124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:38.916988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370308f-fb5d-463c-a8de-7f9119912ec5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:39.010158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:39.010742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.011021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.011292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] PATH_INFO: `/v3/projects/eb36aec3b73c42ef8d610825ccc8ea0c/users/1dfdef53dead4e1abdcfd99b840646fd/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.033464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb36aec3b73c42ef8d610825ccc8ea0c, user_id=1dfdef53dead4e1abdcfd99b840646fd, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:39.034490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:39.034490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:25:39.034490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:25:39.036473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b253bb5-5b52-4a3e-b3da-4a9303e89469 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:39.061333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 748/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:25:38 2022] PUT /identity/v3/projects/eb36aec3b73c42ef8d610825ccc8ea0c/users/1dfdef53dead4e1abdcfd99b840646fd/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:25:39.073525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520cb3-1030-4b4a-a981-75fdc8193ad9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.073813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520cb3-1030-4b4a-a981-75fdc8193ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.074088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520cb3-1030-4b4a-a981-75fdc8193ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.130553 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-28520cb3-1030-4b4a-a981-75fdc8193ad9 None None] MFA Rules not processed for user `1dfdef53dead4e1abdcfd99b840646fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:25:39.131784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-28520cb3-1030-4b4a-a981-75fdc8193ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:39.209153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 754/1502] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:25:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:25:39.222799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c149b4-b79c-47bb-b145-e878ef56aff3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.223088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c149b4-b79c-47bb-b145-e878ef56aff3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.223363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c149b4-b79c-47bb-b145-e878ef56aff3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.224767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 749/1503] 10.222.0.25 () {54 vars in 964 bytes} [Fri Sep 9 06:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:25:39.230327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df271-7bf7-4d5e-8dea-2ee43aaa24e2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.230615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df271-7bf7-4d5e-8dea-2ee43aaa24e2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.230889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df271-7bf7-4d5e-8dea-2ee43aaa24e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.262837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:39.309134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-101df271-7bf7-4d5e-8dea-2ee43aaa24e2 None None] MFA Rules not processed for user `f236664fa09b4a4aafe6980768fc263b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:25:39.310350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-101df271-7bf7-4d5e-8dea-2ee43aaa24e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:39.385482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 755/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Fri Sep 9 06:25:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:25:39.395224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c149b4-b79c-47bb-b145-e878ef56aff3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:39.396784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d9c149b4-b79c-47bb-b145-e878ef56aff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:39.446133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:39.500891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c149b4-b79c-47bb-b145-e878ef56aff3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:39.574889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:39.575521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.575822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.576092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.577669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:39.677543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:39.678979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:39.682463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:39.697905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 750/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:25:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:25:39.743613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48206c19-52fe-45b5-af67-b61dfbe9ee18 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.743906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48206c19-52fe-45b5-af67-b61dfbe9ee18 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.744185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48206c19-52fe-45b5-af67-b61dfbe9ee18 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.745454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 756/1506] 10.222.0.25 () {52 vars in 830 bytes} [Fri Sep 9 06:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:25:39.749236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:39.750460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:39.848176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75044c2-c464-4603-a9cf-ac4ad5844141 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:39.941876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:39.942546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:39.942761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:39.943032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] PATH_INFO: `/v3/users/1dfdef53dead4e1abdcfd99b840646fd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:39.956906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfdef53dead4e1abdcfd99b840646fd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:39.958161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:39.958161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:25:39.958161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:25:39.959371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:39.969720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:40.025501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfdef53dead4e1abdcfd99b840646fd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:40.034683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfdef53dead4e1abdcfd99b840646fd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:40.035214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfdef53dead4e1abdcfd99b840646fd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:40.050741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfdef53dead4e1abdcfd99b840646fd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:40.074694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfdef53dead4e1abdcfd99b840646fd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:40.080241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 751/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:25:39 2022] DELETE /identity/v3/users/1dfdef53dead4e1abdcfd99b840646fd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:25:40.084686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:40.085174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:40.085270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:40.141926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:25:40.143042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:40.217214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:40.229365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 757/1508] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:25:40.238880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:40.240138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:40.341631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad6d5-399e-4621-8b52-d49eed680094 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:40.396515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:40.419243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:40.419887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:40.420167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:40.420440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:40.422065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:40.530398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:40.531892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cab70-404e-46e7-b2c8-3374cc36577b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:40.550874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 752/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:25:40.904324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:25:40.905610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:25:40.917957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9fbcf-8275-4912-82af-339912960280 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:25:41.018736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:25:41.019362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:25:41.019697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:25:41.019959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] PATH_INFO: `/v3/projects/eb36aec3b73c42ef8d610825ccc8ea0c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:25:41.030399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb36aec3b73c42ef8d610825ccc8ea0c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:25:41.031412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:25:41.031412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:25:41.031412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:25:41.032666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:25:41.090406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb36aec3b73c42ef8d610825ccc8ea0c', 'request_id': 'req-04fff7c9-19a2-47f2-aff6-93116bd54b06'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:41.096352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb36aec3b73c42ef8d610825ccc8ea0c', 'request_id': 'req-04fff7c9-19a2-47f2-aff6-93116bd54b06'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:41.106622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb36aec3b73c42ef8d610825ccc8ea0c', 'request_id': 'req-04fff7c9-19a2-47f2-aff6-93116bd54b06'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:41.136580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] The token cache is being invalidate because project eb36aec3b73c42ef8d610825ccc8ea0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:25:41.136863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:41.137282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04fff7c9-19a2-47f2-aff6-93116bd54b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:25:41.138659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 758/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:25:40 2022] DELETE /identity/v3/projects/eb36aec3b73c42ef8d610825ccc8ea0c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:25:50.548564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=62 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:25:51.134260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:27:08.208844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3d682-3f33-46d2-abce-cba500b5f396 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:08.209456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3d682-3f33-46d2-abce-cba500b5f396 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:08.209456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3d682-3f33-46d2-abce-cba500b5f396 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:08.210741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:08.215793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a7051-5a29-4438-a89e-ef39847c18d6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:08.216230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a7051-5a29-4438-a89e-ef39847c18d6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:08.216336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a7051-5a29-4438-a89e-ef39847c18d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:08.296676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-980a7051-5a29-4438-a89e-ef39847c18d6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:08.297884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-980a7051-5a29-4438-a89e-ef39847c18d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:08.399482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:08.554981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3d682-3f33-46d2-abce-cba500b5f396 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:08.557362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-14b3d682-3f33-46d2-abce-cba500b5f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:08.677610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3d682-3f33-46d2-abce-cba500b5f396 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:08.683110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45879-bd0f-4966-83ee-4bb5cda77c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:08.683739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45879-bd0f-4966-83ee-4bb5cda77c87 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:08.684017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45879-bd0f-4966-83ee-4bb5cda77c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:08.684287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45879-bd0f-4966-83ee-4bb5cda77c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:08.692146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45879-bd0f-4966-83ee-4bb5cda77c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:08.693747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45879-bd0f-4966-83ee-4bb5cda77c87 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:08.702955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 754/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:09.800985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057f88f-9081-4ea0-8b98-320bdacde82c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:09.801568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057f88f-9081-4ea0-8b98-320bdacde82c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:09.801568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057f88f-9081-4ea0-8b98-320bdacde82c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:09.802924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:09.809639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e63e-5b6d-4b96-8e06-95adf2e83b40 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:09.810078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e63e-5b6d-4b96-8e06-95adf2e83b40 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:09.810191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42e63e-5b6d-4b96-8e06-95adf2e83b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:09.830480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-be42e63e-5b6d-4b96-8e06-95adf2e83b40 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:09.831884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be42e63e-5b6d-4b96-8e06-95adf2e83b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:09.841213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:09.994021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057f88f-9081-4ea0-8b98-320bdacde82c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:10.007975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057f88f-9081-4ea0-8b98-320bdacde82c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:10.013299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca56b-84ae-4f6a-ba79-07af794995f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:10.013891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca56b-84ae-4f6a-ba79-07af794995f5 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:10.014171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca56b-84ae-4f6a-ba79-07af794995f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:10.014443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca56b-84ae-4f6a-ba79-07af794995f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:10.022387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca56b-84ae-4f6a-ba79-07af794995f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:10.024636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca56b-84ae-4f6a-ba79-07af794995f5 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:10.033986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 761/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:10.338784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f487b-59ab-4e49-a9c8-c96eca9dc0cb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:10.338994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f487b-59ab-4e49-a9c8-c96eca9dc0cb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:10.339265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f487b-59ab-4e49-a9c8-c96eca9dc0cb None None] PATH_INFO: `/v3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:10.340565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 756/1517] 10.222.0.33 () {54 vars in 925 bytes} [Fri Sep 9 06:27:10 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:27:10.349470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadc0-7a17-4ee4-bb82-efde73206fc2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:10.349774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadc0-7a17-4ee4-bb82-efde73206fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:10.350049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadc0-7a17-4ee4-bb82-efde73206fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:10.409586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e1eaadc0-7a17-4ee4-bb82-efde73206fc2 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:10.410789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e1eaadc0-7a17-4ee4-bb82-efde73206fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:10.427185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:27:10.483807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 762/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Sep 9 06:27:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:14.161131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e40b-7f15-4af5-a91e-345b8462bb21 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:14.161565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e40b-7f15-4af5-a91e-345b8462bb21 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:14.161695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e40b-7f15-4af5-a91e-345b8462bb21 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:14.163077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 757/1519] 10.222.0.25 () {54 vars in 961 bytes} [Fri Sep 9 06:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:14.168746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e847-ed70-4c07-afca-5bf4f69a5b82 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:14.169202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e847-ed70-4c07-afca-5bf4f69a5b82 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:14.169307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e847-ed70-4c07-afca-5bf4f69a5b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:14.188906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8418e847-ed70-4c07-afca-5bf4f69a5b82 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:14.190086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8418e847-ed70-4c07-afca-5bf4f69a5b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:14.198497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 763/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Fri Sep 9 06:27:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:14.205973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e40b-7f15-4af5-a91e-345b8462bb21 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:14.219325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e40b-7f15-4af5-a91e-345b8462bb21 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:14.224340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892370e-f3c3-4b27-8137-8db0c622cb0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:14.224928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892370e-f3c3-4b27-8137-8db0c622cb0e service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:14.225211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892370e-f3c3-4b27-8137-8db0c622cb0e service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:14.225484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892370e-f3c3-4b27-8137-8db0c622cb0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:14.234637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892370e-f3c3-4b27-8137-8db0c622cb0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:14.236308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892370e-f3c3-4b27-8137-8db0c622cb0e service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:14.246882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 758/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:27:21.500434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6368-5376-4719-874a-cea82f0c1bbb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:21.501012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6368-5376-4719-874a-cea82f0c1bbb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:21.501012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6368-5376-4719-874a-cea82f0c1bbb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:21.502372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:21.508673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab259b-fb9a-454b-9d6d-37dce24c2995 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:21.509125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab259b-fb9a-454b-9d6d-37dce24c2995 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:21.509232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab259b-fb9a-454b-9d6d-37dce24c2995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:21.528737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5cab259b-fb9a-454b-9d6d-37dce24c2995 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:21.530153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5cab259b-fb9a-454b-9d6d-37dce24c2995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:21.537650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:21.693490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6368-5376-4719-874a-cea82f0c1bbb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:21.708490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6368-5376-4719-874a-cea82f0c1bbb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:21.714627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757a1ce-56fb-4392-a06d-15bf3f35ea34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:21.715218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a1ce-56fb-4392-a06d-15bf3f35ea34 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:21.715495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a1ce-56fb-4392-a06d-15bf3f35ea34 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:21.715792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757a1ce-56fb-4392-a06d-15bf3f35ea34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:21.723724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757a1ce-56fb-4392-a06d-15bf3f35ea34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:21.725293 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757a1ce-56fb-4392-a06d-15bf3f35ea34 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:21.734961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 765/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:22.804410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee14ae-7650-4c82-af02-0f0a542a7c5b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:22.805129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee14ae-7650-4c82-af02-0f0a542a7c5b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:22.805129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee14ae-7650-4c82-af02-0f0a542a7c5b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:22.806515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 06:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:22.812770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55adb59-77f6-4e99-8a05-484f12150157 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:22.813174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55adb59-77f6-4e99-8a05-484f12150157 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:22.813308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55adb59-77f6-4e99-8a05-484f12150157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:22.833930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e55adb59-77f6-4e99-8a05-484f12150157 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:22.835106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e55adb59-77f6-4e99-8a05-484f12150157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:22.843432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 06:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:22.999152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee14ae-7650-4c82-af02-0f0a542a7c5b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:23.013712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee14ae-7650-4c82-af02-0f0a542a7c5b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:23.019051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2527c-7c31-48a8-a22c-0305b8901db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:23.019675 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2527c-7c31-48a8-a22c-0305b8901db9 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:23.019955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2527c-7c31-48a8-a22c-0305b8901db9 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:23.020223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2527c-7c31-48a8-a22c-0305b8901db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:23.028817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2527c-7c31-48a8-a22c-0305b8901db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:23.030392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2527c-7c31-48a8-a22c-0305b8901db9 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:23.040581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 761/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:26.985791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732a1ec-7856-4b66-9f26-bcb5d9490b2e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:26.986385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732a1ec-7856-4b66-9f26-bcb5d9490b2e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:26.986385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732a1ec-7856-4b66-9f26-bcb5d9490b2e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:26.987713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:26.993049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36514d-6969-4969-b49e-4f5ca6f1ecb4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:26.993501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36514d-6969-4969-b49e-4f5ca6f1ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:26.993600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36514d-6969-4969-b49e-4f5ca6f1ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:27.013304 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9c36514d-6969-4969-b49e-4f5ca6f1ecb4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:27.014504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c36514d-6969-4969-b49e-4f5ca6f1ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:27.022399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:27.311647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732a1ec-7856-4b66-9f26-bcb5d9490b2e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:27.313311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9732a1ec-7856-4b66-9f26-bcb5d9490b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:27.391812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732a1ec-7856-4b66-9f26-bcb5d9490b2e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:27.395290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2fba9-8c8e-4887-a65c-d88d57152237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:27.395916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2fba9-8c8e-4887-a65c-d88d57152237 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:27.396194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2fba9-8c8e-4887-a65c-d88d57152237 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:27.396468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2fba9-8c8e-4887-a65c-d88d57152237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:27.405877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2fba9-8c8e-4887-a65c-d88d57152237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:27.407301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2fba9-8c8e-4887-a65c-d88d57152237 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:27.417250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 768/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:29.574370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1feaa5-1a09-4d41-a6d8-26d797656799 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:29.574938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1feaa5-1a09-4d41-a6d8-26d797656799 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:29.574938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1feaa5-1a09-4d41-a6d8-26d797656799 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:29.576330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:29.581778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f07e-f1c6-4bbe-b1cc-f8a3d9541e6a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:29.582177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f07e-f1c6-4bbe-b1cc-f8a3d9541e6a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:29.582456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1f07e-f1c6-4bbe-b1cc-f8a3d9541e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:29.602177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-32e1f07e-f1c6-4bbe-b1cc-f8a3d9541e6a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:29.603288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-32e1f07e-f1c6-4bbe-b1cc-f8a3d9541e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:29.610912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:27:29.907236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1feaa5-1a09-4d41-a6d8-26d797656799 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:29.919792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1feaa5-1a09-4d41-a6d8-26d797656799 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:29.923620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2717b-a0b7-4aa1-96f0-0f24d1c0a936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:29.924227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2717b-a0b7-4aa1-96f0-0f24d1c0a936 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:29.924509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2717b-a0b7-4aa1-96f0-0f24d1c0a936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:29.924784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2717b-a0b7-4aa1-96f0-0f24d1c0a936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:29.934746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2717b-a0b7-4aa1-96f0-0f24d1c0a936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:29.936194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2717b-a0b7-4aa1-96f0-0f24d1c0a936 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:29.946112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 764/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:33.557119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83762666-2559-4493-9da0-0b25d851b492 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:33.557665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83762666-2559-4493-9da0-0b25d851b492 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:33.557665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83762666-2559-4493-9da0-0b25d851b492 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:33.558995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:33.564401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0850fc-1eb2-4637-9e44-d2cef476fd8c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:33.564834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0850fc-1eb2-4637-9e44-d2cef476fd8c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:33.564945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0850fc-1eb2-4637-9e44-d2cef476fd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:33.573588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:27:33.587496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff0850fc-1eb2-4637-9e44-d2cef476fd8c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:33.588762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff0850fc-1eb2-4637-9e44-d2cef476fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:33.596814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:33.902310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83762666-2559-4493-9da0-0b25d851b492 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:33.904004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-83762666-2559-4493-9da0-0b25d851b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:33.968778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:27:33.982696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83762666-2559-4493-9da0-0b25d851b492 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:33.986386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1360a-9e75-4647-874e-98920c615b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:33.986983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1360a-9e75-4647-874e-98920c615b92 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:33.987257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1360a-9e75-4647-874e-98920c615b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:33.987525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1360a-9e75-4647-874e-98920c615b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:33.996892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1360a-9e75-4647-874e-98920c615b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:33.998426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1360a-9e75-4647-874e-98920c615b92 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:34.008226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 771/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:36.156819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25508-c1cd-4fd8-a7ef-474d7ff672d1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:36.157478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25508-c1cd-4fd8-a7ef-474d7ff672d1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:36.157478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25508-c1cd-4fd8-a7ef-474d7ff672d1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:36.158709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 06:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:27:36.164142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1ca2e-a046-4abc-991f-5c9df523648d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:36.164625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1ca2e-a046-4abc-991f-5c9df523648d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:36.164728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1ca2e-a046-4abc-991f-5c9df523648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:36.184022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4ad1ca2e-a046-4abc-991f-5c9df523648d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:27:36.185238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4ad1ca2e-a046-4abc-991f-5c9df523648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:36.193093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 06:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:27:36.501593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25508-c1cd-4fd8-a7ef-474d7ff672d1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:27:36.503283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9a25508-c1cd-4fd8-a7ef-474d7ff672d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:27:36.516809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25508-c1cd-4fd8-a7ef-474d7ff672d1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:27:36.520211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b977b85-edda-40a1-afe4-1dc43c8e7872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:27:36.520795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b977b85-edda-40a1-afe4-1dc43c8e7872 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:27:36.521072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b977b85-edda-40a1-afe4-1dc43c8e7872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:27:36.521342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b977b85-edda-40a1-afe4-1dc43c8e7872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:27:36.530241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b977b85-edda-40a1-afe4-1dc43c8e7872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:27:36.531675 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b977b85-edda-40a1-afe4-1dc43c8e7872 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:27:36.541074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 767/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:27:46.179498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=11 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:27:46.536715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:03.614276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:03.615097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:03.615097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:03.646986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:03.648420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:03.661545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 773/1540] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:03.670582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:03.671141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:03.671141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:03.705079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:03.706333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:03.724781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 768/1541] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:03.730237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:03.746231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779dd0-6023-4eb5-bbaa-a761e98770a8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:03.757087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:03.757853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:03.758125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:03.758392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:03.759062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:03.759416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:03.761166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:03.779069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 774/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:03.787523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:03.803720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06756b6f-5959-46b1-a9a0-991d33828a10 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:03.812656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d82825-0be4-4737-9925-42d8e8f5f940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:03.813654 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82825-0be4-4737-9925-42d8e8f5f940 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:03.814181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82825-0be4-4737-9925-42d8e8f5f940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:03.814674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82825-0be4-4737-9925-42d8e8f5f940 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:03.815533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82825-0be4-4737-9925-42d8e8f5f940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:03.818671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82825-0be4-4737-9925-42d8e8f5f940 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:03.876424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 769/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:03 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:03.881235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:03.884772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f5e8c-aad6-41f5-9a63-dc8c53edef26 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:03.885329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f5e8c-aad6-41f5-9a63-dc8c53edef26 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:03.885834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f5e8c-aad6-41f5-9a63-dc8c53edef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:03.895787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e566c5-49ac-4634-8304-76ad32a466e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:03.918877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ec4f5e8c-aad6-41f5-9a63-dc8c53edef26 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:03.920510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ec4f5e8c-aad6-41f5-9a63-dc8c53edef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.007789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea289bb5-caa5-44c5-b6fd-7af1b1775494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.008411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea289bb5-caa5-44c5-b6fd-7af1b1775494 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.008686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea289bb5-caa5-44c5-b6fd-7af1b1775494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.008957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea289bb5-caa5-44c5-b6fd-7af1b1775494 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.009735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea289bb5-caa5-44c5-b6fd-7af1b1775494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.011863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea289bb5-caa5-44c5-b6fd-7af1b1775494 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.038996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 770/1544] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:04.043422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd476a-dc63-4834-95de-633900b1bbfb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.044065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd476a-dc63-4834-95de-633900b1bbfb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.044582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd476a-dc63-4834-95de-633900b1bbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.050835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 775/1545] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:03 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.052383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.052813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.053352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.085747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.087182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.090420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-40dd476a-dc63-4834-95de-633900b1bbfb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.091969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-40dd476a-dc63-4834-95de-633900b1bbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.107203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 776/1546] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.109899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 771/1547] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.111034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.112577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.114204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.116328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.133768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d4545-59b8-4a67-af5f-8b44fbaa941d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.139385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386e710-7d57-4617-9d07-f7caaf05ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.140169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386e710-7d57-4617-9d07-f7caaf05ea79 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.140526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386e710-7d57-4617-9d07-f7caaf05ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.140891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386e710-7d57-4617-9d07-f7caaf05ea79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.141621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386e710-7d57-4617-9d07-f7caaf05ea79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.143293 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386e710-7d57-4617-9d07-f7caaf05ea79 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.147381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.149010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.159658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 777/1548] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:04.159658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f22766-020e-4ab3-acff-89b11d4599e8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.159658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f22766-020e-4ab3-acff-89b11d4599e8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.159658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f22766-020e-4ab3-acff-89b11d4599e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.162355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 772/1549] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.163569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.174652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8be857-3752-4cd3-8190-f2be1961c3cd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.180350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.181177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.181532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.181874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.182333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e2f22766-020e-4ab3-acff-89b11d4599e8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.182663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.183088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.183959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e2f22766-020e-4ab3-acff-89b11d4599e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.184291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:29:04.184291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:29:04.184291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:29:04.185353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a753fc8-423c-4819-8163-953a7bdfa3fa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.192913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 778/1550] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.194033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.194321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.194598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.207386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 773/1551] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.209896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd577025-dab8-4cef-a31a-bcfad776379c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.210184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd577025-dab8-4cef-a31a-bcfad776379c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.210460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd577025-dab8-4cef-a31a-bcfad776379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.216093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.217414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.226577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 779/1552] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.227463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.232011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fd577025-dab8-4cef-a31a-bcfad776379c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.233202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd577025-dab8-4cef-a31a-bcfad776379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.237083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc1836-e247-474d-a208-a0a3567d2aeb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.241195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 774/1553] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.242502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f21830-3eb2-4ca4-aa12-63bad0569c3c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.242788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f21830-3eb2-4ca4-aa12-63bad0569c3c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.243062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f21830-3eb2-4ca4-aa12-63bad0569c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.246363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657e9cf-933b-405d-9531-fcfaafa6729f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.246979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e9cf-933b-405d-9531-fcfaafa6729f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.247256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e9cf-933b-405d-9531-fcfaafa6729f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.247527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657e9cf-933b-405d-9531-fcfaafa6729f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.248402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657e9cf-933b-405d-9531-fcfaafa6729f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.249769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657e9cf-933b-405d-9531-fcfaafa6729f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.260176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 780/1554] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:04.261579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ec51-18f5-4ad1-ac8b-40770723ff0e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.261860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ec51-18f5-4ad1-ac8b-40770723ff0e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.262131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34ec51-18f5-4ad1-ac8b-40770723ff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.262826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-92f21830-3eb2-4ca4-aa12-63bad0569c3c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.264205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92f21830-3eb2-4ca4-aa12-63bad0569c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.273641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 775/1555] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.274847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.275129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.275399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.282948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ad34ec51-18f5-4ad1-ac8b-40770723ff0e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.284159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ad34ec51-18f5-4ad1-ac8b-40770723ff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.293119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 781/1556] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.294381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.294519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.294700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.294937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.295695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.303841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 776/1557] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.304602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.313633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730da3b4-14c1-4444-ab56-61af73088249 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.314977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.316220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.318661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9218a79-d420-4ed5-94f2-54a4a732c3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.319349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218a79-d420-4ed5-94f2-54a4a732c3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.319660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218a79-d420-4ed5-94f2-54a4a732c3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.319925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218a79-d420-4ed5-94f2-54a4a732c3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.320541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9218a79-d420-4ed5-94f2-54a4a732c3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.322412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9218a79-d420-4ed5-94f2-54a4a732c3f6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.324966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 782/1558] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.325824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.336348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3e1db-9b79-4426-8f7f-b3998e169187 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.341535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.342213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.342577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.342769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.343463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.343794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.345369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.353067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 777/1559] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.354312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.354938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.354938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.358281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 783/1560] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.359022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.369047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e021c-5e84-45c2-9ea6-c2d62604a7f4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.375470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.376698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.425767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.426353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.426626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.426897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] PATH_INFO: `/v3/projects/3d2c1150da924665a18536ea01bfa681/users/a4117df06a874d318819e3aef11d255f/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.429547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:04.431100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:04.442033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 778/1561] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.443411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.448281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2c1150da924665a18536ea01bfa681, user_id=a4117df06a874d318819e3aef11d255f, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.449325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:29:04.449325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:29:04.449325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:29:04.451300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32093b-2a2c-486c-b217-9baf068be47f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.452751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e840c9-0566-444c-97ae-5cbea24dd939 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.457693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.458448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.459323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.459633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.460441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.460798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.462360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.466642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 784/1562] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:04 2022] PUT /identity/v3/projects/3d2c1150da924665a18536ea01bfa681/users/a4117df06a874d318819e3aef11d255f/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:04.477848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.477848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.477848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.478185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 779/1563] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.478185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.484950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d6894-e614-45df-b8cd-dfce68034f68 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.495665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.496874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.542858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.543477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.543797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.544064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.544717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.545023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.546649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b342ef2-eccd-4fcc-b817-ed8a231b09c4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.558966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 780/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.563130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 785/1565] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.563130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.563373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.563373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.563373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.578908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94660570-ef90-4beb-ab5c-479f8c1ce714 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.584232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.584465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.584853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.585133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.585409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.585612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.586040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.588147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.593356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 781/1566] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.594111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.604503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0257e4-77d3-4d28-ae4c-5af4c89524db None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.609184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479418f-bb02-42c2-8949-ab119cefea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.609779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479418f-bb02-42c2-8949-ab119cefea22 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.610061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479418f-bb02-42c2-8949-ab119cefea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.610330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479418f-bb02-42c2-8949-ab119cefea22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.610512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 786/1567] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.610958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479418f-bb02-42c2-8949-ab119cefea22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.612967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479418f-bb02-42c2-8949-ab119cefea22 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.614129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.624051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188d33e-a1c6-46c2-99a8-10608c272341 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.628583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.629191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.629466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.629741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.630986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.631650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.633566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.647299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 782/1568] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.647299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.647299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.647299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.651274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 787/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.652126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.663832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5cc8b-6791-41db-8c39-af7381fe59e7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.680049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] MFA Rules not processed for user `a4117df06a874d318819e3aef11d255f`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:04.681190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:04.720749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.721351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.721626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.721894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.722511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.723537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:29:04.723537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:29:04.723537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:29:04.725339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.745162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 783/1570] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:04.745906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.752618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 788/1571] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.753902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.758707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28deaa-67c6-4bb5-b0d0-d0dfeaeb6625 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.791829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:04.797001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:04.817149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.817789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.818123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.818233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0296-d0eb-47a9-a9ac-55d87592b7e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.818534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.819164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.821105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.823001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.823648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.823929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.824206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.824846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.825179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.826954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.840624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 789/1572] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.841275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.851600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed0b94-8ecb-4d6d-9425-64906ec6910c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.857745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 784/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.857745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.858010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.858010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.858010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.858010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.858374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:04.858374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.859825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.872382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 790/1574] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:04.873404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.883801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e8942-ea60-4b74-ad50-c0d195cba910 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.924453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a9912-3472-4780-b416-b127d072b258 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.929869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.930535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.930751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.931016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.931679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.933258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.938891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.939496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.939808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.940082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.940706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.942602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.943290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 785/1575] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:04.946405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.953595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8a372-3a3e-4e87-a401-7530efca1330 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.959603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.960341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.960759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.961137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.961862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.963896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.968057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 791/1576] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.968057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:04.975305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6091e-e08f-447a-bb24-3ce96420fd03 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:04.980148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:04.980739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:04.981015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:04.981281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:04.981901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:04.983821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:04.993533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 786/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:04.994357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.017833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 792/1578] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.017833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.056812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee46d4b-246c-49ed-9198-bd49971042d4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.079411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19def31-2578-402e-b0c3-ed8315a7ee07 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.085466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.086066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.086346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.086615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.087238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.089167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.098716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:05.116049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.116252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 793/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.116674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.116954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.117232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.118114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.119144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:29:05.119144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:29:05.119144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:29:05.120078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.120383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.134074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:05.143470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 787/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:04 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.144140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.156843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f13a79-785b-43cd-917c-8691683d277e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.185321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc16800-14b8-4b18-ba99-2dfa263ecc7c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.189948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.190558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.190836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.191111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.191767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.194490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.216201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 794/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.216524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.217024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.217134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.217415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.217690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.218329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.220263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.226712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87836d2f-699b-489a-ae0d-3fb85c8e2dd9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.231295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.231903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.232185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.232457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.233104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.234658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.247783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 795/1582] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.248609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.257225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 788/1583] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.257225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.261667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0a1f1-e706-40be-8f51-e19da960b444 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.314762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139740df-0ee8-4acd-8193-304cf831dea4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.316422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.317043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.317316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.317614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.318795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.319983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.320410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.320606 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.320866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.321135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.321762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.323681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.330231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 796/1584] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.330843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.343047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c27f4-b2ad-40f7-915f-eff46b66bed0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.347270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 789/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.348022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.348133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.348688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.349504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.349792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.350511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.352959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.358718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2b827-2f6a-45f6-a73f-76833872b0cc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.363521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.364151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.364434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.364865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.365838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.368393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.375978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 797/1586] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.377132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.389736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042fca-f7ca-4bc7-aa08-bc2af1f9e018 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.391446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 790/1587] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.392360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.398868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.399484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.399797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.400074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.400722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.402320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.403100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8ad1-cd90-4b02-8a9c-1e5c1309829c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.407683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.408277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.408552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.408817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.409450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.411014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.413721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 798/1588] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.414537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.422743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 791/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.423774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.424148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2859044-5ff6-4f22-b4a2-4cd361fabbd0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.429574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.430218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.430493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.430764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] PATH_INFO: `/v3/projects/7a732015d4244c378e0204e1d216f57b/users/1c86ccfec70340c797ecba83163ff1a2/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.434860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04163-226b-499a-98d0-a736429b86eb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.439662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.440198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.440479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.440751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.441378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.443919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.449166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a732015d4244c378e0204e1d216f57b, user_id=1c86ccfec70340c797ecba83163ff1a2, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.451832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.465355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 799/1590] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:05 2022] PUT /identity/v3/projects/7a732015d4244c378e0204e1d216f57b/users/1c86ccfec70340c797ecba83163ff1a2/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:05.466342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.469727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 792/1591] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/users => generated 291 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:05.470659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.476911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:05.483401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f672-d13e-4d05-b873-364ab81f058e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.504998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:05.529646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73cb82-9591-4ba9-965d-c2f18e958a0d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.534542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.535152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.535439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.535824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.536621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.538407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.541963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.542579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.542858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.543131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.543817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.545412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.548887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 800/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.553559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.557478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 793/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.558320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.563939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d4314-8457-4aeb-9ae5-273f6fddbae5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.569089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.569379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9f0fa-7c7d-4d72-bf20-b6b608e8678e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.569694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.569995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.570267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.570902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.572508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.574347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.574955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.575234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.575507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.576183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.577750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.582543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 801/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.583880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.595224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 794/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.595224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.595505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e802ba-c42f-4d2e-b20d-60f2faf00968 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.598206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddb071-03a0-4ad1-8e72-17f2d425b7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.598806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddb071-03a0-4ad1-8e72-17f2d425b7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.599187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddb071-03a0-4ad1-8e72-17f2d425b7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.599584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddb071-03a0-4ad1-8e72-17f2d425b7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.600268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddb071-03a0-4ad1-8e72-17f2d425b7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.602155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddb071-03a0-4ad1-8e72-17f2d425b7a5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.603818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b519e-4264-4622-88da-b66da1c5fe74 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.608351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.608938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.609215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.609487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.610241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.611863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.615854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 802/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.615854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.615854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.615854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.622668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 795/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.623446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.633043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d867a9e-b6e0-4d80-95f0-74b1f524367b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.634976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:05.636199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:05.637904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.638492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.638772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.639046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.639715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.641416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.643909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 803/1598] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:05.644683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.654435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861ffe3-ac43-4e17-b13b-1e29ab89cdc8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.659438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 796/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.659438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.661283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.662503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.662779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.663045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] PATH_INFO: `/v3/projects/e8ee1a9407994bf9a624e6231c7fe565/users/925d5f23bd3f4b35a66736d2a6a3c3eb/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.668533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaafc7-8ab7-45e4-8bcb-bbaf4065d628 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.673062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.673643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.674014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.674182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.674819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.676411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.677550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ee1a9407994bf9a624e6231c7fe565, user_id=925d5f23bd3f4b35a66736d2a6a3c3eb, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.680208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.687282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 797/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.688148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.693245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 804/1601] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:05 2022] PUT /identity/v3/projects/e8ee1a9407994bf9a624e6231c7fe565/users/925d5f23bd3f4b35a66736d2a6a3c3eb/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:05.693904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.700476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e2e9-0dba-42ab-8175-b3c8655a4e7f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.757479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.758146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.758415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc900d59-273d-4549-9054-6e6cf21432c1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.758531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.758715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] PATH_INFO: `/v3/projects/c6b1e971ac3d4176aefaca6b97c946cf/users/0d7bb7805ab8450698c2d9c342c1e496/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.762989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.763619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.763894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.764162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.764794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.766668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.773572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b1e971ac3d4176aefaca6b97c946cf, user_id=0d7bb7805ab8450698c2d9c342c1e496, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.776244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.777208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 805/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.777983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.789765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00e3fd-0787-45c8-a801-aa4f30b7699e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.790731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 798/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:05 2022] PUT /identity/v3/projects/c6b1e971ac3d4176aefaca6b97c946cf/users/0d7bb7805ab8450698c2d9c342c1e496/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:05.791488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.831817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:05.848624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.849301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.849583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.849858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] PATH_INFO: `/v3/projects/7d5de98a77f64cd5beeab6865f5ea7c2/users/0a6757e5408e40cc85ad6145064d8ba7/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.853859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010e9d3-319c-4333-9a76-7913113ae8f0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.858399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.859049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.859326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.859626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] PATH_INFO: `/v3/projects/584cd5d9848d48dea5ac9a01dbaaabf1/users/5f4d9c2505f744b4b62fffac21648f0b/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.865136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5de98a77f64cd5beeab6865f5ea7c2, user_id=0a6757e5408e40cc85ad6145064d8ba7, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.867985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.874833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] RBAC: Authorizing `identity:create_grant(project_id=584cd5d9848d48dea5ac9a01dbaaabf1, user_id=5f4d9c2505f744b4b62fffac21648f0b, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.876492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:05.878201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.886925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 806/1604] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:05 2022] PUT /identity/v3/projects/7d5de98a77f64cd5beeab6865f5ea7c2/users/0a6757e5408e40cc85ad6145064d8ba7/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:05.888020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.892521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 799/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:05 2022] PUT /identity/v3/projects/584cd5d9848d48dea5ac9a01dbaaabf1/users/5f4d9c2505f744b4b62fffac21648f0b/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:05.893192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.943611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047d21a-689b-42dd-9f0b-cc18e7cb6e0c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.954198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2becf3-c5e1-489b-9dc3-d0fb06f0639a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.962730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.963371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.963681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.963968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.964618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:05.966253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:05.977115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 800/1606] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:05.978032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:05.987490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abd671-4c30-4874-9e94-1c450acf5f15 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:05.992593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.993564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.993844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.994113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] PATH_INFO: `/v3/projects/9fae936c35c54adfaf8cb0df7f4ca9dd/users/3e2bdc73390a43ff9eccc57948e2b4a6/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:05.997044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9d150-f012-4b3b-b718-c12dca3c16d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:05.997649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9d150-f012-4b3b-b718-c12dca3c16d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:05.997925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9d150-f012-4b3b-b718-c12dca3c16d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:05.998193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9d150-f012-4b3b-b718-c12dca3c16d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.008351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9d150-f012-4b3b-b718-c12dca3c16d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.009812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9d150-f012-4b3b-b718-c12dca3c16d8 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.010249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fae936c35c54adfaf8cb0df7f4ca9dd, user_id=3e2bdc73390a43ff9eccc57948e2b4a6, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.012905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.021710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 807/1607] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:06.023093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.023383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.023685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.027912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 801/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:05 2022] PUT /identity/v3/projects/9fae936c35c54adfaf8cb0df7f4ca9dd/users/3e2bdc73390a43ff9eccc57948e2b4a6/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:06.031892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.042765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.044303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.101196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8007e2-ed75-4478-a598-c6a1bb6f0027 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.107695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80291b-844f-4a47-ba0d-fb1d8c52d291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.109126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80291b-844f-4a47-ba0d-fb1d8c52d291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.109692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80291b-844f-4a47-ba0d-fb1d8c52d291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.110322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80291b-844f-4a47-ba0d-fb1d8c52d291 admin admin] PATH_INFO: `/v3/projects/9d5d16e92daf44d89d257cf6563d58f4/users/0cb25bf06dcb4e499b5fd49652b0e0a5/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.114930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 808/1609] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.115831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.127863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f597f1-9ecc-451f-845a-271ef7f96519 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.130093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80291b-844f-4a47-ba0d-fb1d8c52d291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5d16e92daf44d89d257cf6563d58f4, user_id=0cb25bf06dcb4e499b5fd49652b0e0a5, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.134074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c11d2-3032-4923-b185-286696cdfd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.134294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80291b-844f-4a47-ba0d-fb1d8c52d291 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.134804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c11d2-3032-4923-b185-286696cdfd3b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.135074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c11d2-3032-4923-b185-286696cdfd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.135390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c11d2-3032-4923-b185-286696cdfd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.136077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c11d2-3032-4923-b185-286696cdfd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.138154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c11d2-3032-4923-b185-286696cdfd3b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.149006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 802/1610] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:06 2022] PUT /identity/v3/projects/9d5d16e92daf44d89d257cf6563d58f4/users/0cb25bf06dcb4e499b5fd49652b0e0a5/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:06.150586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.151005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.151112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 809/1611] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:06.151606 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.152378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62805f6-0c1e-4245-9f0c-9905efca500e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.152666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62805f6-0c1e-4245-9f0c-9905efca500e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.152945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62805f6-0c1e-4245-9f0c-9905efca500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.176724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.178599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d62805f6-0c1e-4245-9f0c-9905efca500e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.179100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.179817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d62805f6-0c1e-4245-9f0c-9905efca500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.189349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:06.236461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:06.249723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 803/1612] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.253943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.257726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 810/1613] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.259442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.259805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.260090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.269070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7355f42-0738-4af9-841a-b0f9eb938576 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.275486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.276188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.276467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.276911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] PATH_INFO: `/v3/projects/9f39103ba8e34405be2e9b45a6c07164/users/03363c2784d44564b167718fb1a7199d/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.280998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.282415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.290903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 811/1614] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.291945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.292887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f39103ba8e34405be2e9b45a6c07164, user_id=03363c2784d44564b167718fb1a7199d, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.296765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.312965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 804/1615] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:06 2022] PUT /identity/v3/projects/9f39103ba8e34405be2e9b45a6c07164/users/03363c2784d44564b167718fb1a7199d/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:06.320628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.372346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca963ba-4b8c-4d98-b5a5-28196769cb58 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.417535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f82238-0891-4868-a47e-f12ec91a6060 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.430317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7af11-a8f5-41a7-9482-82ff29b9e319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.432623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7af11-a8f5-41a7-9482-82ff29b9e319 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.433047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7af11-a8f5-41a7-9482-82ff29b9e319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.433705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7af11-a8f5-41a7-9482-82ff29b9e319 admin admin] PATH_INFO: `/v3/projects/c6b1e971ac3d4176aefaca6b97c946cf/users/0d7bb7805ab8450698c2d9c342c1e496/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.436608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7af11-a8f5-41a7-9482-82ff29b9e319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b1e971ac3d4176aefaca6b97c946cf, user_id=0d7bb7805ab8450698c2d9c342c1e496, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.440055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.440276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7af11-a8f5-41a7-9482-82ff29b9e319 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.440682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.440963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.441239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.456140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.458348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.463644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 805/1616] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:06 2022] PUT /identity/v3/projects/c6b1e971ac3d4176aefaca6b97c946cf/users/0d7bb7805ab8450698c2d9c342c1e496/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:06.465098 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.465449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.465802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.474108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 812/1617] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:06.480916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.484261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.501560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ccdc2-032b-43f7-8e9c-1ad0fb4f3574 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.507732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] MFA Rules not processed for user `0cb25bf06dcb4e499b5fd49652b0e0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.509372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.580666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.582132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.582609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.583449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.594545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 806/1618] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.599858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.616107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246fe53-3bd8-4722-9dd9-fce73d4fa55e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.620869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecff81-1909-4cbf-8b4e-abdfba583a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.621215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecff81-1909-4cbf-8b4e-abdfba583a36 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.621491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecff81-1909-4cbf-8b4e-abdfba583a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.621764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecff81-1909-4cbf-8b4e-abdfba583a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.637618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:06.676635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:06.695915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.697956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.712648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 813/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:06.713693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.719442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecff81-1909-4cbf-8b4e-abdfba583a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.722148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecff81-1909-4cbf-8b4e-abdfba583a36 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.727332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d507db9-cd00-4f45-acf2-af70046d9897 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:06.733139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0601ad8-c9cb-4e7b-8278-fbae97ee3721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:06.733887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0601ad8-c9cb-4e7b-8278-fbae97ee3721 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.734294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0601ad8-c9cb-4e7b-8278-fbae97ee3721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.734802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0601ad8-c9cb-4e7b-8278-fbae97ee3721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.735824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 807/1620] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:06.737443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.737771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.738087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.749493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0601ad8-c9cb-4e7b-8278-fbae97ee3721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:06.752523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0601ad8-c9cb-4e7b-8278-fbae97ee3721 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:06.768139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 814/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:06.778315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:06.779864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:06.779864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:06.792117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] MFA Rules not processed for user `03363c2784d44564b167718fb1a7199d`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.795854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.822735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] MFA Rules not processed for user `0d7bb7805ab8450698c2d9c342c1e496`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:06.824204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:06.964855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 808/1622] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.966322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:06.992145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 815/1623] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:06.994342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.006859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b626-366b-42cd-b4c3-195ca74a0d24 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.011808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be93d5-9a43-48cc-9513-ae65c0d1707d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.019169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.020312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.021229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.021638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.022569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.025614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.026607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.026884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.027604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.028001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.029734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.032904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.076139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 816/1624] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:07.076139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.100612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 809/1625] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:06 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:07.101930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.152407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052b02a-87ee-47f2-8d7d-811044e9df9b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.240616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a24d-9eda-4096-9b03-e1f31a29ed27 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.247136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.247479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.248330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.249062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.249494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.250064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.250372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.250657 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.251192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.253621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.275438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:07.299435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 810/1626] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:07 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:07.299435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.324086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bea1b8-ba67-40f0-9fd0-79460c033803 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.330667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.331430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.331804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.332213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.332888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.335500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.348566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:07.377005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 811/1627] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:07 2022] POST /identity/v3/users => generated 309 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:07.381804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.399395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.401836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.406144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14a897-8ce1-4f70-bc3f-6556fa86c3f1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.418939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 817/1628] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:07.420538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.425325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.427700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.428105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.429142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.432097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.439352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.448037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91e34-e62d-4469-b11d-62495d9a0602 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.455754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.456631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.456869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.457151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.457895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.462466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.486401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 812/1629] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:07.487371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.494128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 818/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:07.498719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.513088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04925da1-a6ca-4755-9123-d392bdd30681 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.524059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1afc3-9b09-49c3-9c99-5a92c9fd069c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.529090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.530232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.530232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.530510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] PATH_INFO: `/v3/projects/b55f39f84ec14a6a95f25900344f0e86/users/d23ce2cc34c345bc9a39325bb620fd84/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.540009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.543104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.544246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.544949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] PATH_INFO: `/v3/projects/e38e6504bd4d4b76abaa2ede6cab147d/users/178780d866c149f79efe4224606dd8c4/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.568665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55f39f84ec14a6a95f25900344f0e86, user_id=d23ce2cc34c345bc9a39325bb620fd84, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.574705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.598325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38e6504bd4d4b76abaa2ede6cab147d, user_id=178780d866c149f79efe4224606dd8c4, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.604679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 813/1631] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:07 2022] PUT /identity/v3/projects/b55f39f84ec14a6a95f25900344f0e86/users/d23ce2cc34c345bc9a39325bb620fd84/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:07.606201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.612704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.644169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 819/1632] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:07 2022] PUT /identity/v3/projects/e38e6504bd4d4b76abaa2ede6cab147d/users/178780d866c149f79efe4224606dd8c4/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:07.645407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:07.774205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873af36-e3a2-486e-909d-356dae1fda42 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.790221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:07.796889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:07.797376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:07.797869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:07.842710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e736611-9750-4f22-b92d-b6905656a2e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:07.942037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:07.943657 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:07.975282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 820/1633] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:07.976558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:08.034114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:08.035128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.035664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.036149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.037294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:08.039224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:08.043748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:08.074437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 814/1634] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:08.077932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:08.101899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ec2aa-b677-49a3-a71d-0a7d8680044c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:08.123782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56aff0b-d31f-4ce9-974d-f588d05574a2 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:08.128733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02fb2e-9065-4241-90c3-7a48d1a18e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:08.129429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02fb2e-9065-4241-90c3-7a48d1a18e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.129886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02fb2e-9065-4241-90c3-7a48d1a18e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.130708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02fb2e-9065-4241-90c3-7a48d1a18e13 admin admin] PATH_INFO: `/v3/projects/b55f39f84ec14a6a95f25900344f0e86/users/d23ce2cc34c345bc9a39325bb620fd84/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.136887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:08.137828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.139024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.139901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.141833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:08.144488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:08.149097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02fb2e-9065-4241-90c3-7a48d1a18e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55f39f84ec14a6a95f25900344f0e86, user_id=d23ce2cc34c345bc9a39325bb620fd84, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:08.160044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02fb2e-9065-4241-90c3-7a48d1a18e13 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:08.170503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 821/1635] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:08.171868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:08.195057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0e700-4907-4df8-a140-c411733f76e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:08.198731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 815/1636] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:08 2022] PUT /identity/v3/projects/b55f39f84ec14a6a95f25900344f0e86/users/d23ce2cc34c345bc9a39325bb620fd84/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:08.209722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.212094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.212591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.303354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] MFA Rules not processed for user `d23ce2cc34c345bc9a39325bb620fd84`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:08.308253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:08.341009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab25e-0f6a-4436-a152-6139bd1a48e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:08.341898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab25e-0f6a-4436-a152-6139bd1a48e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.342288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab25e-0f6a-4436-a152-6139bd1a48e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.343479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab25e-0f6a-4436-a152-6139bd1a48e0 admin admin] PATH_INFO: `/v3/projects/e38e6504bd4d4b76abaa2ede6cab147d/users/178780d866c149f79efe4224606dd8c4/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.348063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cab25e-0f6a-4436-a152-6139bd1a48e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38e6504bd4d4b76abaa2ede6cab147d, user_id=178780d866c149f79efe4224606dd8c4, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:08.356250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cab25e-0f6a-4436-a152-6139bd1a48e0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:08.385469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 822/1637] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:08 2022] PUT /identity/v3/projects/e38e6504bd4d4b76abaa2ede6cab147d/users/178780d866c149f79efe4224606dd8c4/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:08.389671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:08.396833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.397479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.398373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.507473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 816/1638] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:08.513598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] MFA Rules not processed for user `178780d866c149f79efe4224606dd8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:08.515239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:08.520895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:08.656658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317ffc-9077-4253-a0b2-3c59e30b045f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:08.780739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 823/1639] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:08.780739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:08.798306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:08.798306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.798306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.798306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.853033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:08.875852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8f1d-b42c-4103-b892-1a8d1daba2e4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:08.888238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:08.891231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:08.891665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:08.891960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:08.916112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:08.917721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:08.930259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 824/1640] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:08 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:08.979077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:08.980861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:08.993683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 817/1641] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:09.180249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:09.190662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249da1e-1347-4a55-98b2-63ce883bdfd5 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:09.195060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bfe07-329a-480a-beb1-69ddbabc75d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:09.195858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bfe07-329a-480a-beb1-69ddbabc75d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:09.196244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bfe07-329a-480a-beb1-69ddbabc75d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:09.197188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bfe07-329a-480a-beb1-69ddbabc75d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:09.241286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:09.307819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:09.319216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bfe07-329a-480a-beb1-69ddbabc75d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:09.333001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bfe07-329a-480a-beb1-69ddbabc75d0 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:09.340693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c228-bfdc-463b-a9a4-cff216c310ca service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:09.346877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:09.347575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:09.347875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:09.348150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] PATH_INFO: `/v3/users/0cb25bf06dcb4e499b5fd49652b0e0a5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:09.349575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb25bf06dcb4e499b5fd49652b0e0a5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:09.352206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 825/1642] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:09.352435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:09.472601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb25bf06dcb4e499b5fd49652b0e0a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:09.492645 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb25bf06dcb4e499b5fd49652b0e0a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:09.493416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb25bf06dcb4e499b5fd49652b0e0a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:09.502248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb25bf06dcb4e499b5fd49652b0e0a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:09.532550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb25bf06dcb4e499b5fd49652b0e0a5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:09.558467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 818/1643] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:09 2022] DELETE /identity/v3/users/0cb25bf06dcb4e499b5fd49652b0e0a5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:09.562819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:09.563071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:09.563349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:09.605050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:09.606543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:09.790285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 826/1644] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:09.804571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:09.806066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:09.927445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b069-c288-49e7-aadc-5ec57c997430 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:09.939059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:09.939714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:09.940095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:09.940244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:09.960471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:09.962184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:09.985159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 819/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:11.009385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:11.011302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:11.033606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820cd9a-936f-43c9-949e-2e55e2514ab5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:11.050265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:11.054301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:11.056478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:11.056792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] PATH_INFO: `/v3/projects/9d5d16e92daf44d89d257cf6563d58f4` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:11.061143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5d16e92daf44d89d257cf6563d58f4)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:11.082686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:11.229236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5d16e92daf44d89d257cf6563d58f4', 'request_id': 'req-b1f77959-9369-48ca-b744-be6876ca209c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.248367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5d16e92daf44d89d257cf6563d58f4', 'request_id': 'req-b1f77959-9369-48ca-b744-be6876ca209c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.284035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5d16e92daf44d89d257cf6563d58f4', 'request_id': 'req-b1f77959-9369-48ca-b744-be6876ca209c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.313445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] The token cache is being invalidate because project 9d5d16e92daf44d89d257cf6563d58f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:11.313889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.314838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.323010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:11.323659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 827/1646] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:11 2022] DELETE /identity/v3/projects/9d5d16e92daf44d89d257cf6563d58f4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:11.336583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb06f1-f94d-4b46-bc26-710c48de599e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:11.474115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:11.486532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:11.487344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:11.487774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:11.488114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] PATH_INFO: `/v3/users/178780d866c149f79efe4224606dd8c4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:11.491487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178780d866c149f79efe4224606dd8c4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:11.493903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:11.654217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178780d866c149f79efe4224606dd8c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.677815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178780d866c149f79efe4224606dd8c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.679756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178780d866c149f79efe4224606dd8c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.709397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178780d866c149f79efe4224606dd8c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.755684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e0cfa0f-af5c-46bc-b5b2-6cd0f1a42d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178780d866c149f79efe4224606dd8c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:11.763436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 820/1647] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:11 2022] DELETE /identity/v3/users/178780d866c149f79efe4224606dd8c4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:13.032209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:13.033925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:13.055179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:13.063282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f77959-9369-48ca-b744-be6876ca209c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:13.255623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:13.256297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:13.256578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:13.256853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] PATH_INFO: `/v3/projects/e38e6504bd4d4b76abaa2ede6cab147d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:13.263255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38e6504bd4d4b76abaa2ede6cab147d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:13.270946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:13.430405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38e6504bd4d4b76abaa2ede6cab147d', 'request_id': 'req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:13.438889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38e6504bd4d4b76abaa2ede6cab147d', 'request_id': 'req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:13.461316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38e6504bd4d4b76abaa2ede6cab147d', 'request_id': 'req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:13.493823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] The token cache is being invalidate because project e38e6504bd4d4b76abaa2ede6cab147d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:13.494145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:13.494874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9cec51f2-47b9-4b12-8d5b-b3da0b2cbc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:13.504214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 828/1648] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:13 2022] DELETE /identity/v3/projects/e38e6504bd4d4b76abaa2ede6cab147d => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:13.504506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:13.505995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:13.506379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:13.553470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:13.554955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:13.799439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 821/1649] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:13.820025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:13.820305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:13.820586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:13.851658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:13.853641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:13.873176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:13.882248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 829/1650] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:13.906152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a89e60-a7e3-4c1b-8b74-adde9c406783 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:13.911779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:13.912536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:13.912818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:13.913091 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:13.914029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:13.914918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:13.916052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:13.944568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:13.964020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 822/1651] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:13.964513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8d402-98ba-439f-989d-11fb6b47ddae None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:13.971798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:13.972439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:13.973273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:13.974220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:13.975058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:13.977898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:14.039477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 830/1652] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:13 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:14.042031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:14.082785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35d9a-4bae-4861-aad5-a6ba855d0666 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:14.230370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:14.231698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:14.232142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:14.232560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:14.233528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:14.235969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:14.261257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:14.282631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 823/1653] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:14 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:14.287028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:14.306057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715eb4c-dad7-436a-a177-ae78a0aea1df admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:14.326568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:14.328969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:14.329327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:14.329795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:14.331661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:14.336131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:14.367793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:14.369801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 831/1654] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:14.385234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7bb131-63cb-4b69-81cf-dc1ccb398751 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:14.404585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91b4a3-84c2-404b-a113-e5e519f03334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:14.404850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91b4a3-84c2-404b-a113-e5e519f03334 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:14.405133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91b4a3-84c2-404b-a113-e5e519f03334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:14.410738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91b4a3-84c2-404b-a113-e5e519f03334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:14.412802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91b4a3-84c2-404b-a113-e5e519f03334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:14.414443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91b4a3-84c2-404b-a113-e5e519f03334 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:14.457282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 824/1655] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:14.460245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:14.475710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b3f33-73dd-43ec-beaf-e53f4e74274e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:14.485286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:14.488051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:14.490222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:14.490659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] PATH_INFO: `/v3/projects/73649be26f784df3b8adfed9ad4f0a69/users/60bc633c9d724604a4db1a4da0f81f9c/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:14.529209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=73649be26f784df3b8adfed9ad4f0a69, user_id=60bc633c9d724604a4db1a4da0f81f9c, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:14.534833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:14.563537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 832/1656] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:14 2022] PUT /identity/v3/projects/73649be26f784df3b8adfed9ad4f0a69/users/60bc633c9d724604a4db1a4da0f81f9c/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:14.575622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:14.576462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:14.576964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:14.652530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] MFA Rules not processed for user `60bc633c9d724604a4db1a4da0f81f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:14.653914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:14.754257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 825/1657] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:14.756960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:14.810501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:14.947864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb710429-5d39-4171-a73a-bdc791be2ccf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:14.955058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:14.956428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:14.957582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:14.958535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:14.959740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:14.962122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:15.043405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:15.054645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 833/1658] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:14 2022] POST /identity/v3/projects => generated 367 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:15.057825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d750dbd-9bd1-4ac9-8fbf-8e5867a1d9be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:15.180964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:15.182154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:15.182649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:15.183142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:15.199530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:15.201678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:15.241234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 826/1659] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:15 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:15.244500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:15.269026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f4b99-6355-42e9-8baa-7f1573bbcbcc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:15.287685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:15.288732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:15.289281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:15.289808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:15.290707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:15.292775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:15.320466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 834/1660] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:29:15.328088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:15.367307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7870fc0-1b8f-443c-8113-ad68ee21592c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:15.378779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:15.380143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:15.380588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:15.381017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] PATH_INFO: `/v3/projects/cba982f7108f4748befb6d3291d91d00/users/05874fb9c98845b5a1056cd21d8e9b44/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:15.418149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba982f7108f4748befb6d3291d91d00, user_id=05874fb9c98845b5a1056cd21d8e9b44, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:15.426062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:15.457531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 827/1661] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:15 2022] PUT /identity/v3/projects/cba982f7108f4748befb6d3291d91d00/users/05874fb9c98845b5a1056cd21d8e9b44/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:15.462463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:15.490387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5189ea1-da4d-476e-92f9-2d96d7ca5647 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:15.666442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3283-4fe6-4532-bfb0-d479b4eff0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:15.667249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3283-4fe6-4532-bfb0-d479b4eff0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:15.667663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3283-4fe6-4532-bfb0-d479b4eff0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:15.668212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3283-4fe6-4532-bfb0-d479b4eff0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:15.670224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f3283-4fe6-4532-bfb0-d479b4eff0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:15.674959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f3283-4fe6-4532-bfb0-d479b4eff0a7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:15.703301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 835/1662] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:15.704049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:15.724547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb0262-8487-43a9-b3a7-0ee7267e5da1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:15.739527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:15.740573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:15.741161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:15.741593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] PATH_INFO: `/v3/projects/cba982f7108f4748befb6d3291d91d00/users/05874fb9c98845b5a1056cd21d8e9b44/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:15.744741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba982f7108f4748befb6d3291d91d00, user_id=05874fb9c98845b5a1056cd21d8e9b44, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:15.749096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:15.783668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 828/1663] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:15 2022] PUT /identity/v3/projects/cba982f7108f4748befb6d3291d91d00/users/05874fb9c98845b5a1056cd21d8e9b44/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:15.794832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:15.795276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:15.795694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:15.869646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] MFA Rules not processed for user `05874fb9c98845b5a1056cd21d8e9b44`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:15.871274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:16.054661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 836/1664] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:16.066782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:16.069302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:16.130608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:16.194312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca1d39-2514-4c34-9198-22b7fe743a71 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:16.203718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d039-cbb9-4d08-8245-5664b8a32994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:16.204481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d039-cbb9-4d08-8245-5664b8a32994 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:16.204764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d039-cbb9-4d08-8245-5664b8a32994 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:16.205039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d039-cbb9-4d08-8245-5664b8a32994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:16.233557 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d039-cbb9-4d08-8245-5664b8a32994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:16.235180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d039-cbb9-4d08-8245-5664b8a32994 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:16.275018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 829/1665] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:16.365747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:16.367883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:16.389432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe360-3265-4ca6-97e0-62ac0f4f5e88 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:16.398946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda203c5-f527-4d27-aee7-f1f85150797a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:16.399618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda203c5-f527-4d27-aee7-f1f85150797a service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:16.399879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda203c5-f527-4d27-aee7-f1f85150797a service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:16.400140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda203c5-f527-4d27-aee7-f1f85150797a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:16.474336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:16.621230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda203c5-f527-4d27-aee7-f1f85150797a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:16.622740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda203c5-f527-4d27-aee7-f1f85150797a service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:16.665426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 837/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:20.206090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5921b4-9784-4f05-85a7-3bc0baf79298 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:20.206736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5921b4-9784-4f05-85a7-3bc0baf79298 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:20.206736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5921b4-9784-4f05-85a7-3bc0baf79298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:20.320123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9f5921b4-9784-4f05-85a7-3bc0baf79298 None None] MFA Rules not processed for user `3e2bdc73390a43ff9eccc57948e2b4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:20.321526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9f5921b4-9784-4f05-85a7-3bc0baf79298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:20.487903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 830/1667] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:20.499317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8978b0-25cd-47e7-af3f-ce59e2031825 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:20.500249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8978b0-25cd-47e7-af3f-ce59e2031825 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:20.500675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8978b0-25cd-47e7-af3f-ce59e2031825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:20.539260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3c8978b0-25cd-47e7-af3f-ce59e2031825 None None] MFA Rules not processed for user `3e2bdc73390a43ff9eccc57948e2b4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:20.542700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c8978b0-25cd-47e7-af3f-ce59e2031825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:20.569904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 838/1668] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:20.622655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe445f05-c239-4664-8de1-c28da5d9c101 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:20.622951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe445f05-c239-4664-8de1-c28da5d9c101 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:20.623233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe445f05-c239-4664-8de1-c28da5d9c101 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:20.625598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 831/1669] 10.222.0.25 () {54 vars in 962 bytes} [Fri Sep 9 06:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:20.668274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e9d11-f1d7-4375-ab80-66787778a16e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:20.670240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e9d11-f1d7-4375-ab80-66787778a16e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:20.671206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e9d11-f1d7-4375-ab80-66787778a16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:20.818590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3d5e9d11-f1d7-4375-ab80-66787778a16e None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:20.819882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3d5e9d11-f1d7-4375-ab80-66787778a16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:21.026855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 839/1670] 10.222.0.25 () {58 vars in 1059 bytes} [Fri Sep 9 06:29:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:21.036792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe445f05-c239-4664-8de1-c28da5d9c101 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:21.082873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe445f05-c239-4664-8de1-c28da5d9c101 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:21.104382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbb4e9-51b8-4d88-ad2b-8c25ea1bd4b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:21.107253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbb4e9-51b8-4d88-ad2b-8c25ea1bd4b6 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:21.107917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbb4e9-51b8-4d88-ad2b-8c25ea1bd4b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:21.108214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbb4e9-51b8-4d88-ad2b-8c25ea1bd4b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:21.150707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbb4e9-51b8-4d88-ad2b-8c25ea1bd4b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:21.156074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbb4e9-51b8-4d88-ad2b-8c25ea1bd4b6 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:21.198366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 832/1671] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:22.110679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4f21-3e9b-4c58-ab27-2f280bf84c99 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:22.111239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4f21-3e9b-4c58-ab27-2f280bf84c99 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:22.111239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4f21-3e9b-4c58-ab27-2f280bf84c99 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:22.113014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 840/1672] 10.222.0.25 () {54 vars in 912 bytes} [Fri Sep 9 06:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:22.119031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:22.119500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:22.119618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:22.157640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:22.160736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:22.190055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 833/1673] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:29:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:22.199951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b4f21-3e9b-4c58-ab27-2f280bf84c99 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:22.202276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6f8b4f21-3e9b-4c58-ab27-2f280bf84c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:22.360986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:22.560596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144381-6569-42c8-a121-09f09e6c3d7b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:22.561697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b4f21-3e9b-4c58-ab27-2f280bf84c99 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:22.579089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76eff3-bbbb-429a-8f22-6d72f1bb22cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:22.580227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76eff3-bbbb-429a-8f22-6d72f1bb22cc service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:22.583423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:22.585001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76eff3-bbbb-429a-8f22-6d72f1bb22cc service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:22.585868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:22.586381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76eff3-bbbb-429a-8f22-6d72f1bb22cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:22.587537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:22.589423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] PATH_INFO: `/v3/users/60bc633c9d724604a4db1a4da0f81f9c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:22.593935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60bc633c9d724604a4db1a4da0f81f9c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:22.602293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:22.630710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76eff3-bbbb-429a-8f22-6d72f1bb22cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:22.635875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76eff3-bbbb-429a-8f22-6d72f1bb22cc service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:22.684522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 841/1674] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:22.686261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:22.686791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:22.688527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:22.746603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:22.778196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60bc633c9d724604a4db1a4da0f81f9c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:22.788509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] MFA Rules not processed for user `5f4d9c2505f744b4b62fffac21648f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:22.789873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:22.793200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60bc633c9d724604a4db1a4da0f81f9c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:22.794040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60bc633c9d724604a4db1a4da0f81f9c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:22.808432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60bc633c9d724604a4db1a4da0f81f9c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:22.825206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:22.852602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a884e18f-850d-4083-81de-01190b79b2e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60bc633c9d724604a4db1a4da0f81f9c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:22.872639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 834/1675] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:22 2022] DELETE /identity/v3/users/60bc633c9d724604a4db1a4da0f81f9c => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:22.876706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a0020-2854-465a-857c-1eed309fcee6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:22.877117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a0020-2854-465a-857c-1eed309fcee6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:22.877499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a0020-2854-465a-857c-1eed309fcee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:22.920920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5c6a0020-2854-465a-857c-1eed309fcee6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:22.925019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c6a0020-2854-465a-857c-1eed309fcee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.027949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 842/1676] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.029112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:23.034180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.055466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90fcd8-fcca-49bf-97a8-067282ea05e0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:23.126879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 835/1677] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.128487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf078-bd23-468c-b0f5-e260958259cc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.128891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf078-bd23-468c-b0f5-e260958259cc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.129886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf078-bd23-468c-b0f5-e260958259cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.164774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:23.166212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.166632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.167039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] PATH_INFO: `/v3/users/03363c2784d44564b167718fb1a7199d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.169380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03363c2784d44564b167718fb1a7199d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:23.170761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:29:23.170761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Sep 09 06:29:23.170761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: Sep 09 06:29:23.173959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:23.175345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1b4cf078-bd23-468c-b0f5-e260958259cc None None] MFA Rules not processed for user `5f4d9c2505f744b4b62fffac21648f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:23.177235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b4cf078-bd23-468c-b0f5-e260958259cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.323907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03363c2784d44564b167718fb1a7199d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:23.338892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03363c2784d44564b167718fb1a7199d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:23.340341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03363c2784d44564b167718fb1a7199d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:23.342157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03363c2784d44564b167718fb1a7199d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:23.364418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03363c2784d44564b167718fb1a7199d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:23.366086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 836/1678] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.367688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d54b3-0031-4df5-b5ae-0bfd3a4ad885 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.369928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d54b3-0031-4df5-b5ae-0bfd3a4ad885 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.370433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d54b3-0031-4df5-b5ae-0bfd3a4ad885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.409148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 843/1679] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:23 2022] DELETE /identity/v3/users/03363c2784d44564b167718fb1a7199d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:23.409148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:23.412977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.456940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c52d54b3-0031-4df5-b5ae-0bfd3a4ad885 None None] MFA Rules not processed for user `1c86ccfec70340c797ecba83163ff1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:23.458906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c52d54b3-0031-4df5-b5ae-0bfd3a4ad885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.504520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:23.579272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07fb9-f222-4e2e-a233-4d72b4ecff13 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:23.584086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:23.584733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.585015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.585344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.587361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.666695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:23.696857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 837/1680] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.697234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:23.698762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5098f21-c5e4-43b8-bec6-693f2b7910ae None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.699263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5098f21-c5e4-43b8-bec6-693f2b7910ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.699387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290949fb-d0f6-4bf1-a942-8334d1a85ea4 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:23.699836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5098f21-c5e4-43b8-bec6-693f2b7910ae None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.701769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 838/1681] 10.222.0.25 () {54 vars in 962 bytes} [Fri Sep 9 06:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:23.703264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9b683-fe4b-40ba-9a27-f0f962f9cd65 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.705360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9b683-fe4b-40ba-9a27-f0f962f9cd65 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.710806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9b683-fe4b-40ba-9a27-f0f962f9cd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.719148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 844/1682] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:23.719148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20116b6-dc2a-49dd-9655-524632ef59f0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.719148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20116b6-dc2a-49dd-9655-524632ef59f0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.719148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20116b6-dc2a-49dd-9655-524632ef59f0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.719566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 845/1683] 10.222.0.25 () {54 vars in 912 bytes} [Fri Sep 9 06:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:23.720589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.720868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.721143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.749906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:23.751198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.784638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2fb9b683-fe4b-40ba-9a27-f0f962f9cd65 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:23.789245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2fb9b683-fe4b-40ba-9a27-f0f962f9cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.803211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 839/1684] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.805091 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.814088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.814453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.883484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:23.884980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.904136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 840/1685] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:29:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.905521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:23.907132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:23.921566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc4f0a-52ed-4841-b48d-d0468c0b5106 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:23.924766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 846/1686] 10.222.0.25 () {58 vars in 1059 bytes} [Fri Sep 9 06:29:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:23.927320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:23.928220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.928637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.928770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:23.929269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.946314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:23.948368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:23.954686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579ec01-3443-4bff-9d7d-b5abb24f640f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:23.965741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e40a6-a3a1-40aa-8d8d-19f717af48bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:23.966679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e40a6-a3a1-40aa-8d8d-19f717af48bc service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:23.966980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e40a6-a3a1-40aa-8d8d-19f717af48bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:23.968653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e40a6-a3a1-40aa-8d8d-19f717af48bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:23.985215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 841/1687] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:23.987189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:24.007476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e40a6-a3a1-40aa-8d8d-19f717af48bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:24.012811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e40a6-a3a1-40aa-8d8d-19f717af48bc service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:24.027980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86982536-2777-4834-a1c0-b0586d8d579b service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:24.044477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:24.047635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:24.047874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:24.048206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:24.053025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:24.055107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 847/1688] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:24.174978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:24.180706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:24.209694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 842/1689] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:24.733007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e55e73-0b53-4353-97c6-aa24d249931a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:24.733525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e55e73-0b53-4353-97c6-aa24d249931a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:24.733599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e55e73-0b53-4353-97c6-aa24d249931a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:24.735515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 848/1690] 10.222.0.25 () {54 vars in 912 bytes} [Fri Sep 9 06:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:24.745394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:24.747102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:24.756592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:24.756859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:24.757146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:24.775424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3747a8-46ec-4710-948c-6bba13b7a3e9 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:24.794122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:24.794388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:24.795495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:24.795680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:24.795778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:24.796216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] PATH_INFO: `/v3/projects/9f39103ba8e34405be2e9b45a6c07164` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:24.797583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f39103ba8e34405be2e9b45a6c07164)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:24.799519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 06:29:24.799519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Sep 09 06:29:24.799519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: Sep 09 06:29:24.801023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:24.832791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 849/1691] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:29:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:24.845410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:24.847031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:24.952665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f39103ba8e34405be2e9b45a6c07164', 'request_id': 'req-ead629bc-a91e-432e-bd0d-fc3d2020516f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:24.972025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f39103ba8e34405be2e9b45a6c07164', 'request_id': 'req-ead629bc-a91e-432e-bd0d-fc3d2020516f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:24.989188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f39103ba8e34405be2e9b45a6c07164', 'request_id': 'req-ead629bc-a91e-432e-bd0d-fc3d2020516f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.010608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6dc076-594c-447a-a082-e8964d286037 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:25.015517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:25.048424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] The token cache is being invalidate because project 9f39103ba8e34405be2e9b45a6c07164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:25.049954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.051960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.060761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 843/1692] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:24 2022] DELETE /identity/v3/projects/9f39103ba8e34405be2e9b45a6c07164 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:25.062311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:25.065088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.116707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264840a1-297e-433e-8948-7874d5117042 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:25.119075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264840a1-297e-433e-8948-7874d5117042 service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:25.120455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264840a1-297e-433e-8948-7874d5117042 service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:25.120912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264840a1-297e-433e-8948-7874d5117042 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:25.126843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-264840a1-297e-433e-8948-7874d5117042 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.166288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:25.253051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead629bc-a91e-432e-bd0d-fc3d2020516f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:25.263503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:25.264602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:25.265136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:25.266215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] PATH_INFO: `/v3/users/d23ce2cc34c345bc9a39325bb620fd84` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:25.267763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23ce2cc34c345bc9a39325bb620fd84)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:25.271034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:25.314338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264840a1-297e-433e-8948-7874d5117042 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:25.315824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264840a1-297e-433e-8948-7874d5117042 service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:25.377454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 850/1693] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:25.378279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264840a1-297e-433e-8948-7874d5117042 service glance] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:25.381780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-264840a1-297e-433e-8948-7874d5117042 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.413175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23ce2cc34c345bc9a39325bb620fd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.445506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264840a1-297e-433e-8948-7874d5117042 service glance] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:25.446894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23ce2cc34c345bc9a39325bb620fd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.450017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23ce2cc34c345bc9a39325bb620fd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.464088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23ce2cc34c345bc9a39325bb620fd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.479238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:25.479975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:25.480288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:25.480524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:25.490235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.500327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23ce2cc34c345bc9a39325bb620fd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.527301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 844/1694] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:25 2022] DELETE /identity/v3/users/d23ce2cc34c345bc9a39325bb620fd84 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:25.528462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:25.536975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.684644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdea3d0-08a2-413c-a1c6-b29587a8138d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:25.694893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:25.696154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:25.697133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:25.697562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] PATH_INFO: `/v3/projects/73649be26f784df3b8adfed9ad4f0a69` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:25.698915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=73649be26f784df3b8adfed9ad4f0a69)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:25.703024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:25.732671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:25.734546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:25.761924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 851/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:25.762933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:25.777004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.807026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73649be26f784df3b8adfed9ad4f0a69', 'request_id': 'req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.814013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:25.822807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73649be26f784df3b8adfed9ad4f0a69', 'request_id': 'req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.840561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73649be26f784df3b8adfed9ad4f0a69', 'request_id': 'req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.883361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] The token cache is being invalidate because project 73649be26f784df3b8adfed9ad4f0a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:25.883361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.884423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:25.889567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 845/1696] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:25 2022] DELETE /identity/v3/projects/73649be26f784df3b8adfed9ad4f0a69 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:25.899692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:25.907190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:25.939385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:25.981188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e68d3f-66b1-417d-9dc2-2a9704cc854b service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:26.118618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbfd65-3e95-464f-95a7-041fb10bbe0f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:26.138093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:26.138942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.139892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.140617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] PATH_INFO: `/v3/users/05874fb9c98845b5a1056cd21d8e9b44` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:26.146558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] RBAC: Authorizing `identity:delete_user(user_id=05874fb9c98845b5a1056cd21d8e9b44)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:26.148644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:26.208006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:26.208717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.209013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.214996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:26.227256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:26.268570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:26.269306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72787ee-ebde-49e7-a80a-903874025dc7 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:26.304308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 852/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:26.305613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05874fb9c98845b5a1056cd21d8e9b44'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:26.320062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05874fb9c98845b5a1056cd21d8e9b44'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:26.320699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05874fb9c98845b5a1056cd21d8e9b44'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:26.361182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05874fb9c98845b5a1056cd21d8e9b44'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:26.393593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce71e711-d7ce-4835-accf-bae84f00105b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05874fb9c98845b5a1056cd21d8e9b44'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:26.409504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 846/1698] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:25 2022] DELETE /identity/v3/users/05874fb9c98845b5a1056cd21d8e9b44 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:26.492909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24067de-5a73-4297-8bd5-8c1897d1ac79 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.493227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24067de-5a73-4297-8bd5-8c1897d1ac79 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.493470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24067de-5a73-4297-8bd5-8c1897d1ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:26.537190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a24067de-5a73-4297-8bd5-8c1897d1ac79 None None] MFA Rules not processed for user `3e2bdc73390a43ff9eccc57948e2b4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:26.552324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a24067de-5a73-4297-8bd5-8c1897d1ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:26.562057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.562345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.562619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:26.622806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] MFA Rules not processed for user `0a6757e5408e40cc85ad6145064d8ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:26.625182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:26.699127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 853/1699] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:26.700728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcffd2-ad58-471c-8961-63256cef1ae6 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.701015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcffd2-ad58-471c-8961-63256cef1ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.701409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcffd2-ad58-471c-8961-63256cef1ae6 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:26.709031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 854/1700] 10.222.0.25 () {54 vars in 912 bytes} [Fri Sep 9 06:29:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:26.718969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.719253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.719644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:26.749209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:26.761661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:26.765039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:26.857203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 847/1701] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:26.913737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 855/1702] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:29:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:26.946983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:26.948478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:26.975784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed430ab2-ecfe-4af9-b4b7-c6e2ea6c4de3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:26.983090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58037-da92-44d9-b2a0-2ec5917df653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:26.983747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58037-da92-44d9-b2a0-2ec5917df653 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:26.984050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58037-da92-44d9-b2a0-2ec5917df653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:26.984301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58037-da92-44d9-b2a0-2ec5917df653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:27.002860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b58037-da92-44d9-b2a0-2ec5917df653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:27.004953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b58037-da92-44d9-b2a0-2ec5917df653 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:27.037020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 848/1703] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:27.626227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:27.627915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:27.759950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:27.873279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df2a9-66f1-4327-8609-2e7292e86c74 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:27.885946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:27.886623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:27.886912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:27.887168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] PATH_INFO: `/v3/projects/cba982f7108f4748befb6d3291d91d00` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:27.890089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba982f7108f4748befb6d3291d91d00)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:27.895251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:28.020900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba982f7108f4748befb6d3291d91d00', 'request_id': 'req-79758783-cd58-498e-9177-e54ff2583eaa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.034406 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba982f7108f4748befb6d3291d91d00', 'request_id': 'req-79758783-cd58-498e-9177-e54ff2583eaa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.055298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba982f7108f4748befb6d3291d91d00', 'request_id': 'req-79758783-cd58-498e-9177-e54ff2583eaa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.075731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] The token cache is being invalidate because project cba982f7108f4748befb6d3291d91d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:28.076038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.077439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.080907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 856/1704] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:27 2022] DELETE /identity/v3/projects/cba982f7108f4748befb6d3291d91d00 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:28.089771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109609-314a-49a5-aa93-d711aacd1fc4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:28.090774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109609-314a-49a5-aa93-d711aacd1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:28.092826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b109609-314a-49a5-aa93-d711aacd1fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:28.180072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6b109609-314a-49a5-aa93-d711aacd1fc4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:28.181319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b109609-314a-49a5-aa93-d711aacd1fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:28.263328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:28.265368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:28.422885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 849/1705] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:28.431251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:28.431745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:28.432180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:28.490925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:28.495034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:28.507801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79758783-cd58-498e-9177-e54ff2583eaa admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:28.536127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 850/1706] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:28.548512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:28.549796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:28.552013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:28.554505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:28.555085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:28.561464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:28.562961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:28.607704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e5222-02db-4c5f-b001-1007b01f41c9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:28.623120 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:28.624084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:28.624439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:28.624789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] PATH_INFO: `/v3/projects/b55f39f84ec14a6a95f25900344f0e86` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:28.628377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b55f39f84ec14a6a95f25900344f0e86)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:28.635373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:28.738856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:28.743991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:28.765402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:28.779822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 857/1707] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:28.781322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:28.807372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b55f39f84ec14a6a95f25900344f0e86', 'request_id': 'req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.810159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccce0c-44a0-419d-979e-35041e06e4e1 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:28.821781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b55f39f84ec14a6a95f25900344f0e86', 'request_id': 'req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.850192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b55f39f84ec14a6a95f25900344f0e86', 'request_id': 'req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.901437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] The token cache is being invalidate because project b55f39f84ec14a6a95f25900344f0e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:28.903015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.906803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3d9960e-ba93-45ba-a1ba-38e7a75c6876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:28.916353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 851/1708] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:28 2022] DELETE /identity/v3/projects/b55f39f84ec14a6a95f25900344f0e86 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:28.920435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:28.922501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:28.923108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.039775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] MFA Rules not processed for user `925d5f23bd3f4b35a66736d2a6a3c3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:29.041036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:29.079053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:29.084010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.099899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.103098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.108578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:29.118566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:29.127493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d71b04-bb7e-41d5-83dd-930e5ebfe18e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:29.189297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:29.213586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 858/1709] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:29.216448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.220216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.224772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.241053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 852/1710] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:29.242013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:29.245702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:29.300922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525b5fb-9371-4690-b649-c261ffab1a35 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:29.313358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:29.315310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146cb56-dbde-40ff-83f3-fe501ac7b551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:29.315489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:29.315967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146cb56-dbde-40ff-83f3-fe501ac7b551 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.316247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146cb56-dbde-40ff-83f3-fe501ac7b551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.317080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146cb56-dbde-40ff-83f3-fe501ac7b551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.318096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146cb56-dbde-40ff-83f3-fe501ac7b551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:29.324296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146cb56-dbde-40ff-83f3-fe501ac7b551 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:29.338674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 859/1711] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:29.341042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:29.344446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:29.443794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 853/1712] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:29.445842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.446132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.446402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.506409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:29.507877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:29.589575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27849df-7b4f-46bc-91a9-2ddabd283eec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:29.697221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 854/1713] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:29.698588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:29.747902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caff6cd-6c1c-427d-a2c4-f79d998b516e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:29.765670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:29.766420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.770200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.772763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.773632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:29.781138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:29.783718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:29.784129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.784563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.785184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.856023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:29.871618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 855/1714] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:29 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:29.872939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:29.925241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec76159-011b-44eb-b328-88c44da0f74b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:29.937650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:29.938290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:29.938573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:29.938857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:29.940237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:29.940237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:29.945300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:29.985359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 856/1715] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:29.987485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:30.006661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:30.012161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:30.025232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6b46a-2746-42a6-a08e-9e3437028ac9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:30.041253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:30.042847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.043222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.043783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.044651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:30.046480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:30.056322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 860/1716] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:30.058223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:30.063796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 857/1717] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:30.065510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:30.086661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833eb2a-10a8-4077-811e-aef442cbf646 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:30.092888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:30.093786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.095016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.095016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.095992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:30.099744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:30.102386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff3a1-7b11-452d-b35e-46e1f3fcb8e4 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:30.112661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:30.113334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.113723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.114219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.114884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:30.118776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 858/1718] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:30.122862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:30.143721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:30.167345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e254-d952-4416-954a-6ff7f5c5329d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:30.193180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73338065-e28b-41ef-8ebd-f58c449e8591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:30.195442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338065-e28b-41ef-8ebd-f58c449e8591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.197453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338065-e28b-41ef-8ebd-f58c449e8591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.198642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338065-e28b-41ef-8ebd-f58c449e8591 admin admin] PATH_INFO: `/v3/projects/42ef927b521b48b3944ded9ec6aa7c6c/users/af124423443b45bf898728eacc0a6c4a/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.230322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 861/1719] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:30 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:30.234697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:30.260801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73338065-e28b-41ef-8ebd-f58c449e8591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ef927b521b48b3944ded9ec6aa7c6c, user_id=af124423443b45bf898728eacc0a6c4a, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:30.275955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73338065-e28b-41ef-8ebd-f58c449e8591 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:30.319011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 859/1720] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:30 2022] PUT /identity/v3/projects/42ef927b521b48b3944ded9ec6aa7c6c/users/af124423443b45bf898728eacc0a6c4a/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:30.321308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2ea80-ee64-41ca-ada0-fd7a72c8980e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.321593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2ea80-ee64-41ca-ada0-fd7a72c8980e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.321867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2ea80-ee64-41ca-ada0-fd7a72c8980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.378159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c0d2ea80-ee64-41ca-ada0-fd7a72c8980e None None] MFA Rules not processed for user `af124423443b45bf898728eacc0a6c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:30.380751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0d2ea80-ee64-41ca-ada0-fd7a72c8980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:30.503279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:30.564237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322aa8d-f4ef-4dd4-ba0a-e8d0062e99b3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:30.623633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 860/1721] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:30.625703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.625997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.626279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.854400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:30.854400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:30.870877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:30.871532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:30.871842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:30.872123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:30.872766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:30.899178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.024290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 862/1722] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:30 2022] POST /identity/v3/users => generated 294 bytes in 790 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:31.025782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.064302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53afdc7-7040-4aa2-b445-eca800b7c36f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.081654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.082825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.083105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.083559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.084376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.086319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.188443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 861/1723] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:29:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 564 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:31.191809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.200409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 863/1724] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:31 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:31.215415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.242906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d68635-3250-4dd8-8cf2-10c79953f95a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.291929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:31.405023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.405994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.406496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.406924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.409883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb811ad2-2f9d-4f8b-9482-57b0f7d91b16 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.412772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.417144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.417772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.418488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.418818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.419130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.420039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.422588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.447246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 862/1725] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:31.453111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.458164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 864/1726] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:31 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:31.462299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.477706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501372d-e207-468a-8151-ee0b7bfe86b0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.478365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7f5c1-bf71-4e9d-b604-158ca0432e2f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.483429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.484199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.484571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.484942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.485744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.487459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.490000 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.491243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.492915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.493116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.495263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.499170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.509473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 865/1727] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:31.510836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.523625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4043aa-3e87-4aea-bc3c-a38b105263c4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.524336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 863/1728] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:31.527654 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.529106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.530049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.530480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.531065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.531896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.544908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.552649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59b0a-3ff6-4712-8c76-78107ddec43a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.572224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.573470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.576175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.576461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] PATH_INFO: `/v3/projects/a1cc16dd21cc46eabf8fa8c53b2bd9d5/users/d77443b9e96e476abf4c6d1e23ca4697/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.584847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 866/1729] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:31.586092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.616010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cc16dd21cc46eabf8fa8c53b2bd9d5, user_id=d77443b9e96e476abf4c6d1e23ca4697, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.620179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.624052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6b19f-c6d0-4738-8e8a-be584d580676 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:31.637187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e5929-f876-4cd0-b6e4-5502a81bc8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:31.637881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e5929-f876-4cd0-b6e4-5502a81bc8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.638158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e5929-f876-4cd0-b6e4-5502a81bc8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.638428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e5929-f876-4cd0-b6e4-5502a81bc8fd admin admin] PATH_INFO: `/v3/projects/ff4d2f5ecd5342568c99456d1bbe5101/users/5b4d08785be94526983496bc0fe7683f/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.651172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 864/1730] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:31 2022] PUT /identity/v3/projects/a1cc16dd21cc46eabf8fa8c53b2bd9d5/users/d77443b9e96e476abf4c6d1e23ca4697/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:31.652366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:31.662230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:31.672788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e5929-f876-4cd0-b6e4-5502a81bc8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4d2f5ecd5342568c99456d1bbe5101, user_id=5b4d08785be94526983496bc0fe7683f, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:31.676192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e5929-f876-4cd0-b6e4-5502a81bc8fd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:31.707370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:31.731658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 867/1731] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:31 2022] PUT /identity/v3/projects/ff4d2f5ecd5342568c99456d1bbe5101/users/5b4d08785be94526983496bc0fe7683f/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:31.736326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5767-f6c8-42d4-8786-11761a5a6711 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:31.736619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5767-f6c8-42d4-8786-11761a5a6711 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:31.736899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5767-f6c8-42d4-8786-11761a5a6711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:31.812026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-58db5767-f6c8-42d4-8786-11761a5a6711 None None] MFA Rules not processed for user `d77443b9e96e476abf4c6d1e23ca4697`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:31.813632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58db5767-f6c8-42d4-8786-11761a5a6711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:31.847853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009c05-3c76-473f-866f-25b9eeb83f79 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:32.027932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:32.029689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:32.030129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:32.030657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:32.091477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 868/1732] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:32.095154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:32.095505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:32.102885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:32.139523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:32.141386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:32.177652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 865/1733] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:29:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:29:32.178780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:32.204691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] MFA Rules not processed for user `5b4d08785be94526983496bc0fe7683f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:32.205943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:32.294484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:32.327313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:32.353987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d389f-f884-436a-a5fd-b4f53a0ae957 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:32.386248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:32.387443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:32.387749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:32.388026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:32.388676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:32.390591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:32.453029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 869/1734] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:32.453029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:32.484101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e6824-c24a-4a20-8d81-5fbea6bb5eae None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:32.485734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 866/1735] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:32 2022] POST /identity/v3/projects => generated 387 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:32.502219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:32.732582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:32.733222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:32.733527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:32.733875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:32.734548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:32.742771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a0ab7-aaf0-4af4-8994-63d64f6af9d9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:32.744637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:32.767664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:32.768485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:32.768864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:32.769832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:32.770338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:32.774889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:32.867494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 867/1736] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:32 2022] POST /identity/v3/users => generated 300 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:32.872933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:32.886885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 870/1737] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:32 2022] POST /identity/v3/projects => generated 387 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:32.890493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:32.911368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1f06a-17e8-4775-b763-6ba0baec772c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.060519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.061611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.062589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.063268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.064420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.067671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.085506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 868/1738] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:33.137244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:33.164499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1631b-08cc-4e18-a9ab-cc8120ad6fb5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.173639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.174337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.174620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.174903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] PATH_INFO: `/v3/projects/ef6117ac591c46b2a6572d275ba2ed71/users/108874c3dea1458abe5dadd02544085a/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.217469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6117ac591c46b2a6572d275ba2ed71, user_id=108874c3dea1458abe5dadd02544085a, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.222169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.261429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 869/1739] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:33 2022] PUT /identity/v3/projects/ef6117ac591c46b2a6572d275ba2ed71/users/108874c3dea1458abe5dadd02544085a/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:33.264602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72b6ac-f89d-4a82-b8ae-705e88e85f01 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.272893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:33.417767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:33.481795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.482430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.482709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.482981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.488323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.493902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.532079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff36249-dafa-45d7-a8bf-e8c13f4cac26 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.539136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:33.542899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.544023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.544458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.544922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.545729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.550184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 871/1740] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:32 2022] POST /identity/v3/users => generated 300 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:33.552527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.553923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:33.579388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 870/1741] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:33.583828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:33.587190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6160f6c-5b0b-4d0f-ab28-8e7efee120e8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.601181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718b8a-7aee-4083-98e5-6ce7883e9abc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.602640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.603282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.603587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.603872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.604530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.606160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.617348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ee6a8-990a-4f0c-9f97-c39a2006ed04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.618540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ee6a8-990a-4f0c-9f97-c39a2006ed04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.618985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ee6a8-990a-4f0c-9f97-c39a2006ed04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.619460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ee6a8-990a-4f0c-9f97-c39a2006ed04 admin admin] PATH_INFO: `/v3/projects/ef6117ac591c46b2a6572d275ba2ed71/users/108874c3dea1458abe5dadd02544085a/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.626870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 872/1742] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:33.631182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ee6a8-990a-4f0c-9f97-c39a2006ed04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6117ac591c46b2a6572d275ba2ed71, user_id=108874c3dea1458abe5dadd02544085a, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.631673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:33.634639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ee6a8-990a-4f0c-9f97-c39a2006ed04 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.653534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430a75-8e25-43ac-a171-81f937fe1967 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:33.658021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 871/1743] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:33 2022] PUT /identity/v3/projects/ef6117ac591c46b2a6572d275ba2ed71/users/108874c3dea1458abe5dadd02544085a/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:33.669796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.670549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.671017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.742116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] MFA Rules not processed for user `108874c3dea1458abe5dadd02544085a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:33.743773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:33.796191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:33.796896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:33.797361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:33.797655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] PATH_INFO: `/v3/projects/50a9e0eb1d504cb2a1a09179ab48a2a6/users/113758f23d7c40f2a92202ff9fa7fcfa/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:33.849831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a9e0eb1d504cb2a1a09179ab48a2a6, user_id=113758f23d7c40f2a92202ff9fa7fcfa, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:33.853229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:33.898412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 873/1744] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:33 2022] PUT /identity/v3/projects/50a9e0eb1d504cb2a1a09179ab48a2a6/users/113758f23d7c40f2a92202ff9fa7fcfa/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:33.899598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:34.047108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 872/1745] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:34.056771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:34.058425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:34.099252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c9f7c-fa0a-4080-ae01-5dd4ff3a1569 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:34.113596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:34.114254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:34.114783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:34.115067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:34.115885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:34.122634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:34.167014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 874/1746] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:34.171790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:34.194810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499c11b-1166-49a9-a52d-5e2be24475f2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:34.216212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fec35c-d6d6-4511-af87-9a3677e27e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:34.217208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fec35c-d6d6-4511-af87-9a3677e27e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:34.217694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fec35c-d6d6-4511-af87-9a3677e27e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:34.218134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fec35c-d6d6-4511-af87-9a3677e27e2f admin admin] PATH_INFO: `/v3/projects/50a9e0eb1d504cb2a1a09179ab48a2a6/users/113758f23d7c40f2a92202ff9fa7fcfa/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:34.224696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fec35c-d6d6-4511-af87-9a3677e27e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a9e0eb1d504cb2a1a09179ab48a2a6, user_id=113758f23d7c40f2a92202ff9fa7fcfa, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:34.228194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fec35c-d6d6-4511-af87-9a3677e27e2f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:34.285807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 875/1747] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:34 2022] PUT /identity/v3/projects/50a9e0eb1d504cb2a1a09179ab48a2a6/users/113758f23d7c40f2a92202ff9fa7fcfa/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:34.288048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:34.288510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:34.288945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:34.312910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079c73-06d5-439e-b6f5-0829d6c7f3a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:34.373821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] MFA Rules not processed for user `113758f23d7c40f2a92202ff9fa7fcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:34.376250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:34.468918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:34.501117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:34.502241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:34.502684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:34.504881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:34.540207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:34.678298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 876/1748] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:34.729998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:34.734283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:34.756954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 873/1749] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:34.800250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:34.826525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:34.828244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:34.853818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50905937-ccd1-4eb2-8c46-6ca42154fd24 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:34.872380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b851d3-5429-4bb0-8cd1-039ea61b397c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:34.873164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b851d3-5429-4bb0-8cd1-039ea61b397c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:34.873583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b851d3-5429-4bb0-8cd1-039ea61b397c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:34.873730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b851d3-5429-4bb0-8cd1-039ea61b397c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:34.918536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b851d3-5429-4bb0-8cd1-039ea61b397c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:34.927507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b851d3-5429-4bb0-8cd1-039ea61b397c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:34.965988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 874/1750] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:35.022424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e99e7a-d905-4931-ac34-e093a9f781f6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:35.034302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:35.035900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:35.036417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:35.036907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] PATH_INFO: `/v3/users/d77443b9e96e476abf4c6d1e23ca4697` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:35.040483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77443b9e96e476abf4c6d1e23ca4697)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:35.042633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:35.179362 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77443b9e96e476abf4c6d1e23ca4697'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:35.191762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77443b9e96e476abf4c6d1e23ca4697'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:35.192894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77443b9e96e476abf4c6d1e23ca4697'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:35.194291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77443b9e96e476abf4c6d1e23ca4697'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:35.211296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a180e164-21a6-4f90-a3af-ee572e91cd73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77443b9e96e476abf4c6d1e23ca4697'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:35.284700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 877/1751] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:34 2022] DELETE /identity/v3/users/d77443b9e96e476abf4c6d1e23ca4697 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:35.289414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:35.289925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:35.290207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:35.348005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:35.349029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:35.349690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:35.366653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:35.369486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:35.406694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] MFA Rules not processed for user `0a6757e5408e40cc85ad6145064d8ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:35.411347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:35.608528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 875/1752] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:35.624848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:35.628718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:35.673946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 878/1753] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:35.842124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0e5e8-f2be-47cf-a80e-ed43797fbf50 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:35.858198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:35.859289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:35.860468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:35.863121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:35.889168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:35.896811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:35.951841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 876/1754] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:36.650574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:36.652100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:36.660716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:36.807258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402567e-f20a-43b5-85d4-b0a204d754e5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:36.815832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:36.817094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:36.817383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:36.817661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:36.826214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:37.093704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:37.097497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafba5c-e871-48bd-ac9d-f176cbb07d28 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:37.131142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 879/1755] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:37.361650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:37.364706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:37.398970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a5a9f-1400-4a7f-92c3-1cd65a19b7cf service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:37.414593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:37.420901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:37.421289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:37.421680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] PATH_INFO: `/v3/projects/a1cc16dd21cc46eabf8fa8c53b2bd9d5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:37.428731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cc16dd21cc46eabf8fa8c53b2bd9d5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:37.431136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:37.540639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:37.683129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cc16dd21cc46eabf8fa8c53b2bd9d5', 'request_id': 'req-71d2f298-b9c6-4d16-b5d5-d5023683f327'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:37.709912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cc16dd21cc46eabf8fa8c53b2bd9d5', 'request_id': 'req-71d2f298-b9c6-4d16-b5d5-d5023683f327'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:37.734416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cc16dd21cc46eabf8fa8c53b2bd9d5', 'request_id': 'req-71d2f298-b9c6-4d16-b5d5-d5023683f327'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:37.807469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:37.808065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:37.808449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:37.833282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] The token cache is being invalidate because project a1cc16dd21cc46eabf8fa8c53b2bd9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:37.833584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:37.843237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:37.848469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 877/1756] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:37 2022] DELETE /identity/v3/projects/a1cc16dd21cc46eabf8fa8c53b2bd9d5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:37.854195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:37.867892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:37.883327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] MFA Rules not processed for user `1c86ccfec70340c797ecba83163ff1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:37.889252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:38.138087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2f298-b9c6-4d16-b5d5-d5023683f327 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:38.146478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 880/1757] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:38.162690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:38.163564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:38.164162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:38.165130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] PATH_INFO: `/v3/users/108874c3dea1458abe5dadd02544085a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:38.173403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108874c3dea1458abe5dadd02544085a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:38.176879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:38.329529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108874c3dea1458abe5dadd02544085a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:38.345815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108874c3dea1458abe5dadd02544085a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:38.347020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108874c3dea1458abe5dadd02544085a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:38.369852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108874c3dea1458abe5dadd02544085a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:38.418927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bf86e6e4-6f9b-4a43-ab01-ba1aaaa33485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108874c3dea1458abe5dadd02544085a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:38.440688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 878/1758] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:37 2022] DELETE /identity/v3/users/108874c3dea1458abe5dadd02544085a => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:39.715427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:39.740082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8644c8-4a05-4544-9348-f06e35d91f1b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:39.836784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:39.837463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:39.837747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:39.838013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] PATH_INFO: `/v3/projects/ef6117ac591c46b2a6572d275ba2ed71` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:39.840294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6117ac591c46b2a6572d275ba2ed71)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:39.843403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:39.859871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:39.914627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6117ac591c46b2a6572d275ba2ed71', 'request_id': 'req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:39.920679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6117ac591c46b2a6572d275ba2ed71', 'request_id': 'req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:39.929355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6117ac591c46b2a6572d275ba2ed71', 'request_id': 'req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:39.944732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] The token cache is being invalidate because project ef6117ac591c46b2a6572d275ba2ed71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:39.945028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:39.945835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d3a2c323-ddf3-4ed2-b97e-fe1093a0b12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:39.951814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 881/1759] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:39 2022] DELETE /identity/v3/projects/ef6117ac591c46b2a6572d275ba2ed71 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:39.953599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:39.954711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:39.955338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:39.988488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:40.005895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:40.044777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:40.116643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 879/1760] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:40.125671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.125820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.128552 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.152782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:40.154092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:40.163769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 882/1761] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:40.168267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:40.190934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6456e-7088-428d-b23e-1e246809c12d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:40.198292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:40.198922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.199245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.199509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.200213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:40.200517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:40.202118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:40.216295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 880/1762] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:40.220776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:40.230991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26add0e-4f57-46d8-b6f2-fc6438aa6606 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:40.242604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:40.243455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.243786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.246571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.248585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:40.251032 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:40.289003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 883/1763] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:40 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:40.295390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:40.311729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b5cb0-8a49-4f5c-9ed5-04f25e597f44 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:40.405151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:40.407825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.409265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.411087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.413213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:40.416461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:40.455129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 881/1764] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:40 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:40.456867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:40.478010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5eabe-ffee-4476-b6a8-785b860e5abb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:40.485503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:40.486292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.486721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.487134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.488395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:40.490460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:40.531685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 884/1765] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:40.533074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:40.559194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51741b-c721-4aa1-8129-a2d1feef9a04 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:40.571116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a68a6-25de-4fdd-a437-6f6a1d764d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:40.571792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a68a6-25de-4fdd-a437-6f6a1d764d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.573412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a68a6-25de-4fdd-a437-6f6a1d764d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.573412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a68a6-25de-4fdd-a437-6f6a1d764d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.573412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a68a6-25de-4fdd-a437-6f6a1d764d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:40.575232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a68a6-25de-4fdd-a437-6f6a1d764d43 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:40.588559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 882/1766] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:40.592666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:40.606036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bde92-6ad0-42b1-ae8d-aacda5ed8774 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:40.621846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99465e25-505c-4330-b724-1f98f9d6b0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:40.623175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99465e25-505c-4330-b724-1f98f9d6b0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.623954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99465e25-505c-4330-b724-1f98f9d6b0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.625110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99465e25-505c-4330-b724-1f98f9d6b0e9 admin admin] PATH_INFO: `/v3/projects/e3c1141b88a54d8f97ff5b35261c7b28/users/cc15f2dc9c1a40a5b63a5de6c3373790/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.659429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99465e25-505c-4330-b724-1f98f9d6b0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c1141b88a54d8f97ff5b35261c7b28, user_id=cc15f2dc9c1a40a5b63a5de6c3373790, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:40.664854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99465e25-505c-4330-b724-1f98f9d6b0e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:40.716827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 885/1767] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:40 2022] PUT /identity/v3/projects/e3c1141b88a54d8f97ff5b35261c7b28/users/cc15f2dc9c1a40a5b63a5de6c3373790/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:40.720530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.721279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.721903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.797109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] MFA Rules not processed for user `cc15f2dc9c1a40a5b63a5de6c3373790`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:40.808611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:40.831067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:40.831689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:40.832066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:40.903574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] MFA Rules not processed for user `925d5f23bd3f4b35a66736d2a6a3c3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:40.914545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:40.955534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 883/1768] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:40.976856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:41.003029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 886/1769] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:41.034633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:41.098152 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075deac1-0c4b-4683-b68a-ef302887063c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:41.104348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:41.105948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.106349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.106699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.107938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:41.113776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:41.152198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 884/1770] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:40 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:41.156197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:41.177664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302c2d3-1e83-4642-b27d-7b5a8e51f7fd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:41.250523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:41.311610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:41.312618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.312618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.313499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.314393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:41.317454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:41.351535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 887/1771] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:41 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:41.355402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:41.370125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ac7e3-faa3-43e2-b05c-24bd5d1ada72 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:41.382659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:41.383207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.383529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.383858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.384664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:41.386318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:41.399528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 885/1772] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:41.403781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:41.428149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41a80-0675-4e0e-8ff0-649d0651db63 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:41.439130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:41.440164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.440602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.440995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] PATH_INFO: `/v3/projects/bcbc0a4ae1a74c5187d8672e3981396f/users/c96faff971ae446f9bfbf8d747260568/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.464183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbc0a4ae1a74c5187d8672e3981396f, user_id=c96faff971ae446f9bfbf8d747260568, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:41.469221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:41.495451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 888/1773] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:41 2022] PUT /identity/v3/projects/bcbc0a4ae1a74c5187d8672e3981396f/users/c96faff971ae446f9bfbf8d747260568/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:41.500497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:41.515805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c82d-ee2a-4c36-8634-eb1ab9762139 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:41.642276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176bcb9-db1f-483a-85b0-db2580bb156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:41.643491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bcb9-db1f-483a-85b0-db2580bb156a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.644670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bcb9-db1f-483a-85b0-db2580bb156a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.645106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bcb9-db1f-483a-85b0-db2580bb156a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.646113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176bcb9-db1f-483a-85b0-db2580bb156a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:41.648495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176bcb9-db1f-483a-85b0-db2580bb156a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:41.669659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 886/1774] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:41.671299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:41.691840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008122-623d-49b8-bd61-aaeb54458400 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:41.701993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:41.704125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.704401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.704794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] PATH_INFO: `/v3/projects/bcbc0a4ae1a74c5187d8672e3981396f/users/c96faff971ae446f9bfbf8d747260568/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.708689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbc0a4ae1a74c5187d8672e3981396f, user_id=c96faff971ae446f9bfbf8d747260568, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:41.712142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:41.772834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 889/1775] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:41 2022] PUT /identity/v3/projects/bcbc0a4ae1a74c5187d8672e3981396f/users/c96faff971ae446f9bfbf8d747260568/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:41.786510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6066a-8de6-4e1d-8432-947eed7a09ff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:41.786916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6066a-8de6-4e1d-8432-947eed7a09ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:41.787199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6066a-8de6-4e1d-8432-947eed7a09ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:41.846025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e2c6066a-8de6-4e1d-8432-947eed7a09ff None None] MFA Rules not processed for user `c96faff971ae446f9bfbf8d747260568`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:41.849203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e2c6066a-8de6-4e1d-8432-947eed7a09ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.008410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 887/1776] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:42.021785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:42.024180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.107236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e17b5-b0a6-4347-b3a5-a502b67ed497 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:42.112529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:42.114349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:42.114745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:42.115117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:42.127776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:42.129868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:42.148852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 890/1777] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:42.272310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:42.276442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:42.277122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:42.283310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:42.361448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:42.365658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.414687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:42.608761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0561-97ae-4157-a297-24b7886ff8ff service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:42.631871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 888/1778] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Sep 9 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:42.637799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:42.638650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:42.639107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:42.639561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] PATH_INFO: `/v3/users/cc15f2dc9c1a40a5b63a5de6c3373790` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:42.641363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc15f2dc9c1a40a5b63a5de6c3373790)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:42.644523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:42.675280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:42.677765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.717626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82eb1e3-7028-4469-87d3-55eb4ab08006 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:42.736758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:42.736921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:42.737430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:42.737795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:42.742627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:42.774250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc15f2dc9c1a40a5b63a5de6c3373790'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:42.792810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:42.794331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:42.828912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc15f2dc9c1a40a5b63a5de6c3373790'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:42.830310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc15f2dc9c1a40a5b63a5de6c3373790'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:42.832163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc15f2dc9c1a40a5b63a5de6c3373790'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:42.851488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc15f2dc9c1a40a5b63a5de6c3373790'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:42.854546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 889/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:42.855442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:42.858342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.879446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce911-e5ef-45e2-a06f-e4be2319b2ec service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:42.894629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:42.895264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:42.895572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:42.895849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:42.907684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.917804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 891/1780] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:42 2022] DELETE /identity/v3/users/cc15f2dc9c1a40a5b63a5de6c3373790 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:42.919078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:42.924637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:42.936347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:42.940138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5484d20-58a4-4a94-829f-b0632c7f54c6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:42.953279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 890/1781] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:42.956949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623937-5707-4f3b-a9c0-8d05f5f8b827 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:42.957546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623937-5707-4f3b-a9c0-8d05f5f8b827 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:42.957974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623937-5707-4f3b-a9c0-8d05f5f8b827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:43.032682 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-44623937-5707-4f3b-a9c0-8d05f5f8b827 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:43.036222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-44623937-5707-4f3b-a9c0-8d05f5f8b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:43.221938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def6ecb-9aa3-45c4-831b-d46b62d60d39 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:43.235769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:43.239927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:43.240216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:43.240496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:43.263462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:43.271364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:43.297602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 892/1782] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:43.328365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 891/1783] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:43.508001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:43.526970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b75a8-1af6-46a7-a6df-ce46e7586e73 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:43.543986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8b773-14b8-4b48-b173-c18a95d0b4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:43.544897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8b773-14b8-4b48-b173-c18a95d0b4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:43.545183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8b773-14b8-4b48-b173-c18a95d0b4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:43.545459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8b773-14b8-4b48-b173-c18a95d0b4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:43.560308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8b773-14b8-4b48-b173-c18a95d0b4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:43.562442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8b773-14b8-4b48-b173-c18a95d0b4b1 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:43.585748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 893/1784] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:45.346580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbffbd-2217-4f27-8bb8-8059b599ffe0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:45.347283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbffbd-2217-4f27-8bb8-8059b599ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:45.347283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbffbd-2217-4f27-8bb8-8059b599ffe0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:45.359962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 892/1785] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:29:45 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:45.403536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:45.404187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:45.404369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:45.495469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:45.497208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:45.580536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbffbd-2217-4f27-8bb8-8059b599ffe0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:45.582935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fecbffbd-2217-4f27-8bb8-8059b599ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:45.756944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 894/1786] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:29:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:45.765306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbffbd-2217-4f27-8bb8-8059b599ffe0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:45.770640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:45.771896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:45.772439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:45.772902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:45.775421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:45.984074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:45.986457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:45.992037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:46.005261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:46.006840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:46.059046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 893/1787] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:46.070815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642ff8-96bc-4c59-b6d3-5bb32a4ee7ab None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:46.099465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:46.103377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:46.120508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:46.120838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:46.180531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:46.196097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:46.250607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 895/1788] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:47.823107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:47.825027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:47.866960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f323550-f4d3-49e5-b125-9c81a9b05415 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:47.884659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:47.887352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:47.888919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:47.889906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] PATH_INFO: `/v3/projects/e3c1141b88a54d8f97ff5b35261c7b28` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:47.894538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c1141b88a54d8f97ff5b35261c7b28)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:47.900862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:48.074219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c1141b88a54d8f97ff5b35261c7b28', 'request_id': 'req-10bf056d-2a8e-4cf9-8849-18f144a4781b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.093516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c1141b88a54d8f97ff5b35261c7b28', 'request_id': 'req-10bf056d-2a8e-4cf9-8849-18f144a4781b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.151791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c1141b88a54d8f97ff5b35261c7b28', 'request_id': 'req-10bf056d-2a8e-4cf9-8849-18f144a4781b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.200672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] The token cache is being invalidate because project e3c1141b88a54d8f97ff5b35261c7b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:48.201130 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.205420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-10bf056d-2a8e-4cf9-8849-18f144a4781b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.211159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 894/1789] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:47 2022] DELETE /identity/v3/projects/e3c1141b88a54d8f97ff5b35261c7b28 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:48.213136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:48.234147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25479745-94af-45a3-a300-62f62a7a65ec service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:48.279608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:48.440442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:48.441626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:48.441964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:48.442351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] PATH_INFO: `/v3/users/c96faff971ae446f9bfbf8d747260568` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:48.449149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96faff971ae446f9bfbf8d747260568)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:48.451429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:48.599691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96faff971ae446f9bfbf8d747260568'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.621579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96faff971ae446f9bfbf8d747260568'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.622342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96faff971ae446f9bfbf8d747260568'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.623233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96faff971ae446f9bfbf8d747260568'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.640778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e7967dfd-0bfc-40f6-9c47-fd59968abc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96faff971ae446f9bfbf8d747260568'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:48.690969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 896/1790] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:48 2022] DELETE /identity/v3/users/c96faff971ae446f9bfbf8d747260568 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:48.963603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec308c0-bdfd-45c0-90ea-6be40ea22821 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:48.965123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec308c0-bdfd-45c0-90ea-6be40ea22821 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:48.965562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec308c0-bdfd-45c0-90ea-6be40ea22821 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:48.967386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 895/1791] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:29:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:29:49.021751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:49.022048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:49.022323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:49.063975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:49.066655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:49.220574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 897/1792] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:29:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:49.701255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec308c0-bdfd-45c0-90ea-6be40ea22821 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:49.705179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bec308c0-bdfd-45c0-90ea-6be40ea22821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:49.812462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec308c0-bdfd-45c0-90ea-6be40ea22821 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:49.820799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:49.821700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:49.822803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:49.823094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] PATH_INFO: `/v3/users/af124423443b45bf898728eacc0a6c4a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:49.824546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af124423443b45bf898728eacc0a6c4a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:49.826758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:49.899171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af124423443b45bf898728eacc0a6c4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:49.905251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:49.907784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:49.913760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af124423443b45bf898728eacc0a6c4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:49.915061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af124423443b45bf898728eacc0a6c4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:49.931781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2787989-e93c-4085-bed0-4d6e933fd735 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:49.939143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af124423443b45bf898728eacc0a6c4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:49.952772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:49.953416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:49.953690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:49.958043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:49.962374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:49.980601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86d5b84c-4e97-4c98-bf8e-e02c6b4b01a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af124423443b45bf898728eacc0a6c4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:49.986197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:49.997836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:50.005407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:50.007827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 896/1793] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:49 2022] DELETE /identity/v3/users/af124423443b45bf898728eacc0a6c4a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:50.012277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.013033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.013587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.018085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 898/1794] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:50.040298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:50.041878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.113676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 897/1795] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:50.121853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.122431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.159649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:50.239954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77e682-9818-4269-aacf-bcaa8dcc1cd8 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:50.302090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de987f28-e150-4f4b-bc13-ca3a0054c2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:50.303283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987f28-e150-4f4b-bc13-ca3a0054c2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.303601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987f28-e150-4f4b-bc13-ca3a0054c2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.304023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987f28-e150-4f4b-bc13-ca3a0054c2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.313984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de987f28-e150-4f4b-bc13-ca3a0054c2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:50.329935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de987f28-e150-4f4b-bc13-ca3a0054c2d5 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:50.337128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 899/1796] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:50.341362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.343242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.379656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5ac05-c8f8-4d0a-9f5b-92466a39d6e3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:50.393554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:50.394471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.394773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.395047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] PATH_INFO: `/v3/projects/bcbc0a4ae1a74c5187d8672e3981396f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.396329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbc0a4ae1a74c5187d8672e3981396f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:50.398307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:50.464858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbc0a4ae1a74c5187d8672e3981396f', 'request_id': 'req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.500616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbc0a4ae1a74c5187d8672e3981396f', 'request_id': 'req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.508767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbc0a4ae1a74c5187d8672e3981396f', 'request_id': 'req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.545814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] The token cache is being invalidate because project bcbc0a4ae1a74c5187d8672e3981396f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:50.545814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.545814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-31bd05a3-c15e-4b5e-bc9e-0ac993fae407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.545814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 898/1797] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:50 2022] DELETE /identity/v3/projects/bcbc0a4ae1a74c5187d8672e3981396f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:50.550514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.550813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.551115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.579846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:50.581957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.657329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 900/1798] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:50.659456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.660242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.660701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.687051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:50.688587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.706641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 899/1799] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:50.707229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.708184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.709246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.719640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af993a8-8a3d-424a-a5ae-4212a5d589f1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:50.725024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:50.727687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b90c1-c806-445a-b777-344f86d12ecc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:50.728610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.729347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.729974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.730906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:50.731620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:50.732808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:50.733545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.733800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.734081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] PATH_INFO: `/v3/projects/42ef927b521b48b3944ded9ec6aa7c6c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.735105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:50.735266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ef927b521b48b3944ded9ec6aa7c6c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:50.737842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:50.758189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 900/1800] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:50.763876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.788004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcd532-a423-441d-84f0-787097758c37 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:50.806543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ef927b521b48b3944ded9ec6aa7c6c', 'request_id': 'req-233f36f5-1f1c-4927-8e3d-f2f397aec18e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.814019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ef927b521b48b3944ded9ec6aa7c6c', 'request_id': 'req-233f36f5-1f1c-4927-8e3d-f2f397aec18e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.824191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ef927b521b48b3944ded9ec6aa7c6c', 'request_id': 'req-233f36f5-1f1c-4927-8e3d-f2f397aec18e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.840497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:50.845890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] The token cache is being invalidate because project 42ef927b521b48b3944ded9ec6aa7c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:50.846324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.848479 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:50.860757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 901/1801] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:50 2022] DELETE /identity/v3/projects/42ef927b521b48b3944ded9ec6aa7c6c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:50.864720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.866698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.888162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:50.888795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:50.889079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:50.889354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:50.890002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:50.891986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:50.898251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:50.944111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 901/1802] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:50 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:50.949600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:50.951617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:50.980653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f36f5-1f1c-4927-8e3d-f2f397aec18e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.056320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2cb85-787b-4f5f-bec2-38f11111ca06 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.062275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.062938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.063205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.063348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.063456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.064138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.064293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.064537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.064896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] PATH_INFO: `/v3/users/5b4d08785be94526983496bc0fe7683f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.067458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.069277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4d08785be94526983496bc0fe7683f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.071643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.105253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 902/1803] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:50 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:51.106473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.119815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c04bb-f65c-4f5d-860b-1d5d84c63dbf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.127344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.128634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.128994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.129348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.130082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.131968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.149421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 903/1804] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:51.151480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.164191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f5d20-9836-4da6-9e46-8651de9518f0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.169821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4d08785be94526983496bc0fe7683f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.170142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.170843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.171238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.171652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.173572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.176150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.180589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4d08785be94526983496bc0fe7683f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.181374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4d08785be94526983496bc0fe7683f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.182323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4d08785be94526983496bc0fe7683f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.193777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 904/1805] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:51.193991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-416045c9-fa00-4b9b-b446-24b421980b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4d08785be94526983496bc0fe7683f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.196494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.202889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:51.221428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4889f-7b1e-468d-aa35-1542b9509b00 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.221760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 902/1806] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:50 2022] DELETE /identity/v3/users/5b4d08785be94526983496bc0fe7683f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:51.307687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.308531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.308820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.309100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] PATH_INFO: `/v3/projects/a2c3401555714ce2b80c6464690da9fd/users/ff62becf806d456a8ee423fe352fa37c/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.326166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c3401555714ce2b80c6464690da9fd, user_id=ff62becf806d456a8ee423fe352fa37c, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.329088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.349018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 905/1807] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:51 2022] PUT /identity/v3/projects/a2c3401555714ce2b80c6464690da9fd/users/ff62becf806d456a8ee423fe352fa37c/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:51.362736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.363404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.363404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.439236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] MFA Rules not processed for user `ff62becf806d456a8ee423fe352fa37c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:51.441508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:51.454721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:51.537233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 903/1808] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:51.542048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.562997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.565972 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:51.606548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:51.649031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d07ba5-ca7e-4c23-9e5d-3d5ff724f6c5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.659911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.660634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.660927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.661206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.661850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.663812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.673074 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c22976-8915-4bff-9037-a46fe30147ae None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.678644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.679277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.679582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.679936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] PATH_INFO: `/v3/projects/ff4d2f5ecd5342568c99456d1bbe5101` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.681090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4d2f5ecd5342568c99456d1bbe5101)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.683248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.696969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 906/1809] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:51 2022] POST /identity/v3/projects => generated 389 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:51.702301 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.736808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4d2f5ecd5342568c99456d1bbe5101', 'request_id': 'req-5205ba77-e01d-487b-a511-bc0fd0af6d96'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.743192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4d2f5ecd5342568c99456d1bbe5101', 'request_id': 'req-5205ba77-e01d-487b-a511-bc0fd0af6d96'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.751699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4d2f5ecd5342568c99456d1bbe5101', 'request_id': 'req-5205ba77-e01d-487b-a511-bc0fd0af6d96'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.764997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] The token cache is being invalidate because project ff4d2f5ecd5342568c99456d1bbe5101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:51.765289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.765957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.767483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 904/1810] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:51 2022] DELETE /identity/v3/projects/ff4d2f5ecd5342568c99456d1bbe5101 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:51.771962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.773325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:51.786502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dded05-2445-499e-a9eb-f0e3627c73e6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.821298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:51.843724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ba77-e01d-487b-a511-bc0fd0af6d96 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.844062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.844898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.845016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.845272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.845974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.848078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.849309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.849975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.850511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.850788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] PATH_INFO: `/v3/users/113758f23d7c40f2a92202ff9fa7fcfa` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.852425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=113758f23d7c40f2a92202ff9fa7fcfa)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.854577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.874189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 907/1811] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:51 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:51.875507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.877269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:51.892977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e5396-c803-46c6-b2cc-b1993857fa6e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.899076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.899715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.899989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.900255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.900934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.902660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.911227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113758f23d7c40f2a92202ff9fa7fcfa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.915608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 908/1812] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:29:51.917923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.920890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113758f23d7c40f2a92202ff9fa7fcfa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.921664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113758f23d7c40f2a92202ff9fa7fcfa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.922414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113758f23d7c40f2a92202ff9fa7fcfa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.928805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150d333-cb49-468b-a470-5e5c8bfd2c1b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:51.931072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113758f23d7c40f2a92202ff9fa7fcfa'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:51.933583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:51.934227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:51.934509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:51.934785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] PATH_INFO: `/v3/projects/f80dbc8cdf24422abfb0d05c25ca3d00/users/da374c559a174a0b8063f8e55b32d030/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:51.950373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 905/1813] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:51 2022] DELETE /identity/v3/users/113758f23d7c40f2a92202ff9fa7fcfa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:51.950563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80dbc8cdf24422abfb0d05c25ca3d00, user_id=da374c559a174a0b8063f8e55b32d030, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:51.953474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:51.967141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 909/1814] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:51 2022] PUT /identity/v3/projects/f80dbc8cdf24422abfb0d05c25ca3d00/users/da374c559a174a0b8063f8e55b32d030/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:51.971233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:51.972746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.041344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64306b0-ec6f-472a-88db-fc6d48301dca admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.104599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbcc13-573c-4ef3-8630-338a8b79a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.105428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbcc13-573c-4ef3-8630-338a8b79a268 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.105804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbcc13-573c-4ef3-8630-338a8b79a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.106147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbcc13-573c-4ef3-8630-338a8b79a268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.106868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbcc13-573c-4ef3-8630-338a8b79a268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.108766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbcc13-573c-4ef3-8630-338a8b79a268 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.119836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 906/1815] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:52.123973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.134597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40807b68-736f-4499-9a57-e96c1f98bbbf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.139193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.139954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.140179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.140451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] PATH_INFO: `/v3/projects/f80dbc8cdf24422abfb0d05c25ca3d00/users/da374c559a174a0b8063f8e55b32d030/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.142382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80dbc8cdf24422abfb0d05c25ca3d00, user_id=da374c559a174a0b8063f8e55b32d030, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.145631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.157877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 910/1816] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:52 2022] PUT /identity/v3/projects/f80dbc8cdf24422abfb0d05c25ca3d00/users/da374c559a174a0b8063f8e55b32d030/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:52.170425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.170716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.171193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.187189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:52.204690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] MFA Rules not processed for user `da374c559a174a0b8063f8e55b32d030`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:52.207003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.263521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.266913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.297682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 907/1817] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:52.298810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:52.307505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.310382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.370209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e7689-f264-4a4b-86aa-d8e5021b286e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.378807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.378807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.378807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.378807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] PATH_INFO: `/v3/projects/50a9e0eb1d504cb2a1a09179ab48a2a6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.379821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a9e0eb1d504cb2a1a09179ab48a2a6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.382441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.384669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67c72b-caba-4ad8-a986-e9fbe645dc4d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.391208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c458be2-23a5-434b-aece-78e0bb0473df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.391208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458be2-23a5-434b-aece-78e0bb0473df service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.391208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458be2-23a5-434b-aece-78e0bb0473df service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.391208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c458be2-23a5-434b-aece-78e0bb0473df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.402265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c458be2-23a5-434b-aece-78e0bb0473df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.403801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c458be2-23a5-434b-aece-78e0bb0473df service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.416534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 908/1818] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:52.453649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a9e0eb1d504cb2a1a09179ab48a2a6', 'request_id': 'req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.466825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a9e0eb1d504cb2a1a09179ab48a2a6', 'request_id': 'req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.476737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a9e0eb1d504cb2a1a09179ab48a2a6', 'request_id': 'req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.498844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] The token cache is being invalidate because project 50a9e0eb1d504cb2a1a09179ab48a2a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:52.499207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.500132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.502043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 911/1819] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:52 2022] DELETE /identity/v3/projects/50a9e0eb1d504cb2a1a09179ab48a2a6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:52.508802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa41a9d-4809-4755-888b-6c49dcb5e8df None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.509176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa41a9d-4809-4755-888b-6c49dcb5e8df None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.509456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa41a9d-4809-4755-888b-6c49dcb5e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.538937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3aa41a9d-4809-4755-888b-6c49dcb5e8df None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:52.540939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3aa41a9d-4809-4755-888b-6c49dcb5e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.598982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.600689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.620540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 909/1820] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:52.628895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.629183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.629456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.648691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:52.649897 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.662084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 910/1821] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:52.667803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.679351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e137f-e102-4bc0-b3c9-cffa2f568a24 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.683572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2574a-41d9-44ca-9d57-34fe44bdd4de None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.691165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.692120 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.692120 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.692436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] PATH_INFO: `/v3/users/ff62becf806d456a8ee423fe352fa37c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.693511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.694126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.694402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.694670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.695314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:52.695645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.697234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.699493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff62becf806d456a8ee423fe352fa37c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.701451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.715987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 911/1822] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:52.720337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.730937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d49a-ad52-4615-ba0c-29bd0a6c5aaf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.735974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.736503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.736793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.737700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.738439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.740613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.764982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:52.766493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:52.768633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff62becf806d456a8ee423fe352fa37c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.777852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff62becf806d456a8ee423fe352fa37c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.779161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff62becf806d456a8ee423fe352fa37c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.786421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 912/1823] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:52 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:52.788034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.789336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff62becf806d456a8ee423fe352fa37c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.790448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.807619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5ace42df-e3f3-4dc1-a644-a98aa43a7323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff62becf806d456a8ee423fe352fa37c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:52.820378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 912/1824] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:52 2022] DELETE /identity/v3/users/ff62becf806d456a8ee423fe352fa37c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:52.824711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.825113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.825518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.845851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:52.847320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.869200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f558733-e33b-4bca-8996-662880956b9b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:52.931849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 913/1825] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:52.949496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:52.950897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:52.953901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:52.954524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:52.954809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:52.955084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:52.956326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:52.958257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:52.989750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 913/1826] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:52 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:52.993764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.005048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e36b86-4d6f-467d-860b-a1b724f62c7d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.010308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.011002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.011357 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.011737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.012531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.015877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.021386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315426b7-ba79-412d-9a23-3bfc9bb98f87 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.026012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.026631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 914/1827] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:29:53.026753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.026958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.027299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.030314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.037781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.039209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.040805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983402be-9c75-4c07-be0a-eff57b455b84 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.045400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d228df1-d0c4-4663-bb75-beedd7dfdc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.045991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d228df1-d0c4-4663-bb75-beedd7dfdc39 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.046273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d228df1-d0c4-4663-bb75-beedd7dfdc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.046545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d228df1-d0c4-4663-bb75-beedd7dfdc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.047228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d228df1-d0c4-4663-bb75-beedd7dfdc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.048781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d228df1-d0c4-4663-bb75-beedd7dfdc39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.049179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 914/1828] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:53.059630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 915/1829] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:29:53.063812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.074380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba874c-9079-4008-b2ef-09477a33c553 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.079378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.080079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.080359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.080632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] PATH_INFO: `/v3/projects/98234d2dd533485293b60619a05da4ae/users/af0dc1f21371470790dce614e38700b2/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.097747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98234d2dd533485293b60619a05da4ae, user_id=af0dc1f21371470790dce614e38700b2, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.101123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.124447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 915/1830] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:53 2022] PUT /identity/v3/projects/98234d2dd533485293b60619a05da4ae/users/af0dc1f21371470790dce614e38700b2/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:53.138476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.138759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.139030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.173527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] MFA Rules not processed for user `af0dc1f21371470790dce614e38700b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:53.175124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:53.253436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:53.264040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 916/1831] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:53.266054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.317903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:53.352383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.354075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:53.359954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db619ea-e059-485c-b8ea-5f10e741708a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.370558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0312-e03b-4063-8226-9f32f99a0045 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.375979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.376860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.377248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.378194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.378951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.379369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.380157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.380327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.380609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] PATH_INFO: `/v3/projects/a2c3401555714ce2b80c6464690da9fd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.381336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.381793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c3401555714ce2b80c6464690da9fd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.384181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.422996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 916/1832] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:53 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:53.424995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.449439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c3401555714ce2b80c6464690da9fd', 'request_id': 'req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.457769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c3401555714ce2b80c6464690da9fd', 'request_id': 'req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.469460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c3401555714ce2b80c6464690da9fd', 'request_id': 'req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.484448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] The token cache is being invalidate because project a2c3401555714ce2b80c6464690da9fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:53.484728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.485476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.488075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 917/1833] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:53 2022] DELETE /identity/v3/projects/a2c3401555714ce2b80c6464690da9fd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:53.492122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.493515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:53.509461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2abc-5fc2-4f02-ad5f-69e28041e8d9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.627053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.628360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.628661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.628907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.629617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.631572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.675727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 917/1834] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:53 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:53.687780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.692823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08f93-8c72-42b9-aeec-efe3fd5cc8ab admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.693830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:53.704402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.705473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.706841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.707723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] PATH_INFO: `/v3/users/da374c559a174a0b8063f8e55b32d030` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.709668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da374c559a174a0b8063f8e55b32d030)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.712497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.712879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389da8-69b6-4ef7-8818-da173e8da551 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.727990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.730095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.730767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.731529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.733608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.739687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.760544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 918/1835] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:53.764712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.778934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dad5-86ab-4527-9dc2-2d4e60cacd55 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:53.785887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:53.786687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:53.787138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:53.787577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] PATH_INFO: `/v3/projects/abc0986c0daa4984924da74fe1e35b84/users/b368b0bc50084ded91e6515215c90d5f/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:53.832615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc0986c0daa4984924da74fe1e35b84, user_id=b368b0bc50084ded91e6515215c90d5f, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:53.836549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:53.840161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da374c559a174a0b8063f8e55b32d030'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.860160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da374c559a174a0b8063f8e55b32d030'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.861983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da374c559a174a0b8063f8e55b32d030'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.863246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da374c559a174a0b8063f8e55b32d030'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.866349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 919/1836] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:53 2022] PUT /identity/v3/projects/abc0986c0daa4984924da74fe1e35b84/users/b368b0bc50084ded91e6515215c90d5f/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:53.869533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:53.872330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:53.883751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da374c559a174a0b8063f8e55b32d030'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:53.927642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:53.937837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 918/1837] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:53 2022] DELETE /identity/v3/users/da374c559a174a0b8063f8e55b32d030 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:53.948703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:54.013106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9549-bf8f-41af-9c23-da520128478a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:54.104528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5cac6-9450-430b-b0ca-de5cdcfbd294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:54.105791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5cac6-9450-430b-b0ca-de5cdcfbd294 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:54.106283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5cac6-9450-430b-b0ca-de5cdcfbd294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:54.106788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5cac6-9450-430b-b0ca-de5cdcfbd294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:54.107789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5cac6-9450-430b-b0ca-de5cdcfbd294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:54.109954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5cac6-9450-430b-b0ca-de5cdcfbd294 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:54.128845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 920/1838] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:29:54.130067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:54.141699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d1155-211e-45d1-bc1f-9362a1d83596 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:54.152417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:54.153123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:54.153398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:54.154061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] PATH_INFO: `/v3/projects/abc0986c0daa4984924da74fe1e35b84/users/b368b0bc50084ded91e6515215c90d5f/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:54.159923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc0986c0daa4984924da74fe1e35b84, user_id=b368b0bc50084ded91e6515215c90d5f, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:54.164930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:54.195857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 919/1839] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:54 2022] PUT /identity/v3/projects/abc0986c0daa4984924da74fe1e35b84/users/b368b0bc50084ded91e6515215c90d5f/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:54.199421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:54.199760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:54.200005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:54.234117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] MFA Rules not processed for user `b368b0bc50084ded91e6515215c90d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:54.235346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:54.335266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 921/1840] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:54.343795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:54.346317 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:54.452098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37756c8-d6a4-4db5-8cb2-565485aa30e7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:54.456420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:54.457376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:54.457720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:54.458060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:54.467904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:54.469467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:54.484877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 920/1841] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:54.594093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:54.608169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e9b9c-ff0d-481b-9260-01de4a1710da None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:54.614565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:54.615285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:54.615684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:54.616036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:54.619136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:54.694904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:54.700997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:54.702508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:54.717853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 922/1842] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:55.754974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:55.756457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:55.870512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0902b-6a2d-43fd-9009-5c5145d5d499 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:55.876710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:55.877454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:55.878284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:55.878741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] PATH_INFO: `/v3/users/a4117df06a874d318819e3aef11d255f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:55.880121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4117df06a874d318819e3aef11d255f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:55.883791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:55.920265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:55.922159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:55.953389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713f4c-d759-4910-b24c-a5b7559d795b service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:55.966434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:55.968708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:55.971962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:55.971962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] PATH_INFO: `/v3/projects/f80dbc8cdf24422abfb0d05c25ca3d00` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:55.973870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80dbc8cdf24422abfb0d05c25ca3d00)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:55.976523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:55.986002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4117df06a874d318819e3aef11d255f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.006436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4117df06a874d318819e3aef11d255f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.007101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4117df06a874d318819e3aef11d255f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.007793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4117df06a874d318819e3aef11d255f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.022485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cdd16708-3b81-44d4-8966-dbdfd9f5f353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4117df06a874d318819e3aef11d255f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.082738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 921/1843] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:55 2022] DELETE /identity/v3/users/a4117df06a874d318819e3aef11d255f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:56.085232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.085771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.088748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.108609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80dbc8cdf24422abfb0d05c25ca3d00', 'request_id': 'req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.128048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80dbc8cdf24422abfb0d05c25ca3d00', 'request_id': 'req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.143904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:56.144800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80dbc8cdf24422abfb0d05c25ca3d00', 'request_id': 'req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.145421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:56.152839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:56.179039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] The token cache is being invalidate because project f80dbc8cdf24422abfb0d05c25ca3d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:56.179700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.180644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a8bcb48-10d1-4830-a414-bfe6fae4fe73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:56.182910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 923/1844] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:55 2022] DELETE /identity/v3/projects/f80dbc8cdf24422abfb0d05c25ca3d00 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:56.188296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8c06e-e94c-4083-a911-6f4da574ec60 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.189649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8c06e-e94c-4083-a911-6f4da574ec60 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.192073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8c06e-e94c-4083-a911-6f4da574ec60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.234057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9da8c06e-e94c-4083-a911-6f4da574ec60 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:56.236522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9da8c06e-e94c-4083-a911-6f4da574ec60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:56.350513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 922/1845] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:56.369543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.371067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:56.423621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 924/1846] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:56.432586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.432957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.433372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.462214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:56.464221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:56.479103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 925/1847] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:56.480173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.492094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dcf09-f245-4c9f-bad5-a95e08aac17a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.498922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.499801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.500396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.500760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.504555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:56.505011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.506775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.511843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33207ff-27d2-4578-8a49-035fcb829e5d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.516881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.518761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.519393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.519863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.526756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 926/1848] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:29:56.527844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.537822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.539210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43835a62-7a74-4957-945f-13217f5eef2c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.541344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.557417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.559043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.560543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.560932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.561689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.564089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 923/1849] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:29:56.566039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.617364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 927/1850] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:56 2022] POST /identity/v3/projects => generated 409 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:56.633980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.647987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee37b6-ac83-4dd2-b90a-5176aa0487bf service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.730361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.731126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.731257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.731470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.732142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.734075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.770589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 924/1851] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:56 2022] POST /identity/v3/users => generated 305 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:56.775305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.797032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e320-9ddd-4141-bc8e-d06e13bfeb97 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.804721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.805438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.806229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.806731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.807656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.809479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.821469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 928/1852] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:56.825396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.838563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5ca8-3f2f-4f6e-ae85-137475a68d56 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.843255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1c89-4da3-4b38-8623-81e11e0da94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.843924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1c89-4da3-4b38-8623-81e11e0da94d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.844206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1c89-4da3-4b38-8623-81e11e0da94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.844478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1c89-4da3-4b38-8623-81e11e0da94d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.845119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1c89-4da3-4b38-8623-81e11e0da94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.846812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1c89-4da3-4b38-8623-81e11e0da94d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.859600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 925/1853] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:29:56.862240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:56.872779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bbe27-71a3-4304-804b-07b72fcd546a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:56.878010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:56.878816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.879156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.879493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] PATH_INFO: `/v3/projects/cb6b2759b80f4cfa93052c4cac85d957/users/0e77e728f7c84d09917246ca5efcf6fd/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.895028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b2759b80f4cfa93052c4cac85d957, user_id=0e77e728f7c84d09917246ca5efcf6fd, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:56.898002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:56.907322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:56.922230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 929/1854] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:56 2022] PUT /identity/v3/projects/cb6b2759b80f4cfa93052c4cac85d957/users/0e77e728f7c84d09917246ca5efcf6fd/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:56.924872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:56.925213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:56.925571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:56.957870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] MFA Rules not processed for user `0e77e728f7c84d09917246ca5efcf6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:56.959094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:56.980876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:57.032519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 926/1855] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:56 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:57.034309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.151396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0799823-0981-4311-adb2-e010a66f3c0b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.161330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.162033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.162322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.162640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.163297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.166060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.208351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 930/1856] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:57 2022] POST /identity/v3/projects => generated 409 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:57.209952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.225449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f76f1d-89ca-4481-8c22-8977c966bc98 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.331290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.332252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.332780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.333136 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.334023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.336289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.368224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 927/1857] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:57 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:29:57.372441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.391030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14c6e4-88a2-45cd-bdc0-561533d81824 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.398254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.399012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.399308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.399717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.400393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.402423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.420400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 931/1858] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:57.425284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.435804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eabf9a-f501-4bea-8c45-6c9969ee7f45 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.440936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.441614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.441896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.442181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] PATH_INFO: `/v3/projects/2d00cd83b768420694904bce12b8f5aa/users/ddb9df0aed9e456e940453aab684717e/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.459835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d00cd83b768420694904bce12b8f5aa, user_id=ddb9df0aed9e456e940453aab684717e, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.463983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.488082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 928/1859] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:57 2022] PUT /identity/v3/projects/2d00cd83b768420694904bce12b8f5aa/users/ddb9df0aed9e456e940453aab684717e/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:57.489410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.501845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef041673-fc70-496f-a640-79115b967f7b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.599266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.600303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.601454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.603458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:57.603671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.603671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.603989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.608272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.616382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970a659-a75a-4762-b857-292d97d0334d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.623243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.624066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.624449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.624835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] PATH_INFO: `/v3/users/af0dc1f21371470790dce614e38700b2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.626411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0dc1f21371470790dce614e38700b2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.632056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.640353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 932/1860] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:29:57.642648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.676222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdaf14-29e5-4e7f-9c10-a72cae100cf6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:57.714357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:57.716073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.716589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.717418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] PATH_INFO: `/v3/projects/2d00cd83b768420694904bce12b8f5aa/users/ddb9df0aed9e456e940453aab684717e/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.721450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d00cd83b768420694904bce12b8f5aa, user_id=ddb9df0aed9e456e940453aab684717e, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:57.731465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:57.752999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 933/1861] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:29:57 2022] PUT /identity/v3/projects/2d00cd83b768420694904bce12b8f5aa/users/ddb9df0aed9e456e940453aab684717e/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:57.756264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:57.759529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:57.766067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0dc1f21371470790dce614e38700b2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:57.786209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0dc1f21371470790dce614e38700b2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:57.787917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0dc1f21371470790dce614e38700b2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:57.788780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0dc1f21371470790dce614e38700b2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:57.801874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-992ede4d-9c4a-4dd2-a7ea-84f7195d7678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0dc1f21371470790dce614e38700b2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:57.833039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:57.844299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 929/1862] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:57 2022] DELETE /identity/v3/users/af0dc1f21371470790dce614e38700b2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:29:57.847072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e6c70-0dac-412a-9f08-efc36b300c24 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:57.847936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e6c70-0dac-412a-9f08-efc36b300c24 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:57.848368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e6c70-0dac-412a-9f08-efc36b300c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:57.898743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-360e6c70-0dac-412a-9f08-efc36b300c24 None None] MFA Rules not processed for user `ddb9df0aed9e456e940453aab684717e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:57.900692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-360e6c70-0dac-412a-9f08-efc36b300c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:57.926858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:57.931819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e9ea1-3b97-4ee6-b201-e407e10df700 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:58.010052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:58.011204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:58.011765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:58.012161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] PATH_INFO: `/v3/projects/3d2c1150da924665a18536ea01bfa681` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:58.014722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 930/1863] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:57 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:58.014859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2c1150da924665a18536ea01bfa681)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:58.017288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:58.017865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:58.018207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:58.018964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:58.044289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:58.046894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.059082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 931/1864] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:58.060403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:58.063152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.086413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2c1150da924665a18536ea01bfa681', 'request_id': 'req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.101134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2c1150da924665a18536ea01bfa681', 'request_id': 'req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.109667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2c1150da924665a18536ea01bfa681', 'request_id': 'req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.127465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] The token cache is being invalidate because project 3d2c1150da924665a18536ea01bfa681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:58.127816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.129295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 934/1865] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:57 2022] DELETE /identity/v3/projects/3d2c1150da924665a18536ea01bfa681 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:58.135899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:58.145402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.161379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a941f47-173e-4123-8ea8-bea7ad80af4e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:58.232616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e548a0-31ec-47e9-a40a-01a47beaf8b3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:58.233063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:58.234941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:58.235413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:58.235953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:58.238583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:58.238746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.239209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:58.239487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:58.239791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:58.241550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.303530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:58.305904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:58.332229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:58.333639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:58.344215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:58.347168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:58.347377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 935/1866] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:58.348196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:58.350246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.363447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 932/1867] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:58 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:58.364884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6fa7b-5d94-403f-9a3b-40b2355bf8b4 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:58.372670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:58.373498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:58.373885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:58.374284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] PATH_INFO: `/v3/users/0d7bb7805ab8450698c2d9c342c1e496` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:58.375722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7bb7805ab8450698c2d9c342c1e496)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:58.378002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:58.482108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7bb7805ab8450698c2d9c342c1e496'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.505188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7bb7805ab8450698c2d9c342c1e496'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.506014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7bb7805ab8450698c2d9c342c1e496'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.521589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7bb7805ab8450698c2d9c342c1e496'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.545783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7bb7805ab8450698c2d9c342c1e496'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:58.562456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 936/1868] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:58 2022] DELETE /identity/v3/users/0d7bb7805ab8450698c2d9c342c1e496 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:58.652155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:58.653616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.665802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0036594-9c2f-429b-839d-7543ca4bb142 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:58.725252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:58.725902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:58.726872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:58.726872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:58.728643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:58.797765 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:58.799368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:58.809407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 933/1869] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:29:58 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:29:58.969970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:58.972329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:59.161775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:59.163220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:59.196701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196104c-d272-43b9-9a56-be03225237fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:59.198655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14900c-f3bc-4088-8cfb-1a6edbfd1800 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:59.206099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:59.207232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.207611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.207960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] PATH_INFO: `/v3/projects/98234d2dd533485293b60619a05da4ae` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.209234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98234d2dd533485293b60619a05da4ae)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:59.209718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:59.210584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.211635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.211635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] PATH_INFO: `/v3/projects/c6b1e971ac3d4176aefaca6b97c946cf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.212011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:59.212877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b1e971ac3d4176aefaca6b97c946cf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:59.214919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:59.288338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98234d2dd533485293b60619a05da4ae', 'request_id': 'req-1c873a6f-2543-4473-b0a9-4ee57b176495'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.298869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98234d2dd533485293b60619a05da4ae', 'request_id': 'req-1c873a6f-2543-4473-b0a9-4ee57b176495'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.315771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:59.316976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98234d2dd533485293b60619a05da4ae', 'request_id': 'req-1c873a6f-2543-4473-b0a9-4ee57b176495'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.329822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b1e971ac3d4176aefaca6b97c946cf', 'request_id': 'req-a21ea195-f687-4aa6-a81d-6f24b5f757b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.347436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b1e971ac3d4176aefaca6b97c946cf', 'request_id': 'req-a21ea195-f687-4aa6-a81d-6f24b5f757b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.349003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] The token cache is being invalidate because project 98234d2dd533485293b60619a05da4ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:59.349728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.352181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.357527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 937/1870] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:58 2022] DELETE /identity/v3/projects/98234d2dd533485293b60619a05da4ae => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:59.359802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:59.370306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:59.379152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b1e971ac3d4176aefaca6b97c946cf', 'request_id': 'req-a21ea195-f687-4aa6-a81d-6f24b5f757b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.416101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] The token cache is being invalidate because project c6b1e971ac3d4176aefaca6b97c946cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:29:59.419335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.419335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a21ea195-f687-4aa6-a81d-6f24b5f757b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.419335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 934/1871] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:29:59 2022] DELETE /identity/v3/projects/c6b1e971ac3d4176aefaca6b97c946cf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:59.424203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14413-0361-4fae-ac8a-8e90c7932f4a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.425721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14413-0361-4fae-ac8a-8e90c7932f4a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.426874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14413-0361-4fae-ac8a-8e90c7932f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.483358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a6f14413-0361-4fae-ac8a-8e90c7932f4a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:59.487716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a6f14413-0361-4fae-ac8a-8e90c7932f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:59.517604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:59.543643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c873a6f-2543-4473-b0a9-4ee57b176495 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:59.635868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 935/1872] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:29:59.638106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.639054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.639448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.661085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:59.662133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.662418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.662756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] PATH_INFO: `/v3/users/b368b0bc50084ded91e6515215c90d5f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.664748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b368b0bc50084ded91e6515215c90d5f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:59.666950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:59.668570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:29:59.670356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:59.679774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 936/1873] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:29:59.684631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:59.694828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10424-412d-491b-8676-32bc39bf0997 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:59.700172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:59.700852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.701134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.701411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.702187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:29:59.702621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:59.704346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:59.719584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 937/1874] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:29:59.723992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:59.729308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b368b0bc50084ded91e6515215c90d5f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.734342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5b4b-d7a3-4345-993d-ff6ee08ee2be admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:59.739526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b368b0bc50084ded91e6515215c90d5f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.740291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b368b0bc50084ded91e6515215c90d5f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.741279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:59.741901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.742182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.742456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.743093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:59.745216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:29:59.749020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b368b0bc50084ded91e6515215c90d5f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.765916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b368b0bc50084ded91e6515215c90d5f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:29:59.774906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 938/1875] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:29:59 2022] DELETE /identity/v3/users/b368b0bc50084ded91e6515215c90d5f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:29:59.784160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 938/1876] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:29:59 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:29:59.800647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:29:59.802260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:29:59.823118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:29:59.905959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be7e0f-902b-4828-82aa-9d46f8151893 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:29:59.989382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:29:59.990065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:29:59.990376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:29:59.990521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:29:59.991243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:29:59.993756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.023742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 939/1877] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:29:59 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:00.025973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.038503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534068d-7f85-479a-9bfb-9a3c23bb1dde admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.043889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.044506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.044847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.045829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.046482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.048313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.064041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 939/1878] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:00.065081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.076697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269878f5-f2b0-4abd-9c7d-8c7f2f5e7ee6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.081932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.082658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.083061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.083465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.084696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.085958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.099097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 940/1879] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:00.102550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.104295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.105350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:00.120684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbac4fa-3a0c-4d70-8321-bda66b0cfec6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.122342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96869a30-9198-4b30-825a-d2e8f163ac56 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.125890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6808bd-87e2-4d62-8dc7-8936608c7edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.126739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6808bd-87e2-4d62-8dc7-8936608c7edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.127830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6808bd-87e2-4d62-8dc7-8936608c7edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.128163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6808bd-87e2-4d62-8dc7-8936608c7edb admin admin] PATH_INFO: `/v3/projects/1b32e00055b44fd585972c3444160fb7/users/76dae40f0e834a2290a508f14472fad0/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.129225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.130104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.130459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.130811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] PATH_INFO: `/v3/projects/abc0986c0daa4984924da74fe1e35b84` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.134643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc0986c0daa4984924da74fe1e35b84)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.137159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.147878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6808bd-87e2-4d62-8dc7-8936608c7edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b32e00055b44fd585972c3444160fb7, user_id=76dae40f0e834a2290a508f14472fad0, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.150567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6808bd-87e2-4d62-8dc7-8936608c7edb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.172059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 941/1880] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:00 2022] PUT /identity/v3/projects/1b32e00055b44fd585972c3444160fb7/users/76dae40f0e834a2290a508f14472fad0/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:00.179699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.180149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.180515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.212398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc0986c0daa4984924da74fe1e35b84', 'request_id': 'req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:00.220010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc0986c0daa4984924da74fe1e35b84', 'request_id': 'req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:00.238026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc0986c0daa4984924da74fe1e35b84', 'request_id': 'req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:00.238470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] MFA Rules not processed for user `76dae40f0e834a2290a508f14472fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:00.242786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:00.255810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] The token cache is being invalidate because project abc0986c0daa4984924da74fe1e35b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:00.256107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:00.256865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3ae8235d-7f98-4a36-b241-5f265c8d1c0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:00.259374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 940/1881] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:00 2022] DELETE /identity/v3/projects/abc0986c0daa4984924da74fe1e35b84 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:00.264220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988ca68-0745-4a8a-bc21-32850f5e0419 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.264620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988ca68-0745-4a8a-bc21-32850f5e0419 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.264891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988ca68-0745-4a8a-bc21-32850f5e0419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.289054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1988ca68-0745-4a8a-bc21-32850f5e0419 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:00.290211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1988ca68-0745-4a8a-bc21-32850f5e0419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:00.331381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 942/1882] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:00.335364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.336731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:00.341068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:00.342626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:00.358275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 941/1883] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:00.367403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.367771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.368019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.387707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:00.388929 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:00.397690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 942/1884] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:00.401761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.411517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df94c-1c9e-4648-a080-88592561b69c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.415477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94a73a-2bc5-4617-9e73-581bca913d4d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.416494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.417141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.417418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.417690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.418404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:00.418646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.420267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.420485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065942c-bb88-4481-9742-440232026acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.421090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065942c-bb88-4481-9742-440232026acf admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.422873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065942c-bb88-4481-9742-440232026acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.422873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065942c-bb88-4481-9742-440232026acf admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.422873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065942c-bb88-4481-9742-440232026acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.425246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065942c-bb88-4481-9742-440232026acf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.438937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 943/1885] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:30:00.440108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.450709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf5062-9d3f-4c3b-90b8-4f13c1beb85d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.460254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.460648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.460928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.461198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.461831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.463753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.466417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 943/1886] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:00.470271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065942c-bb88-4481-9742-440232026acf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.516723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 944/1887] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/projects => generated 359 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:00.522552 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.583720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065942c-bb88-4481-9742-440232026acf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.613189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea1975-250a-4bc8-8f7d-351ea4b417ce admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.622558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.623502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.624130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.624587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.626325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.630584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.659631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 945/1888] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/users => generated 280 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:00.663598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.667600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.668458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.668744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.669219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.670138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.672581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.680449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ca72-eaf7-4896-9af5-547401d97268 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.687818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.688605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.688885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.689213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.690842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.693522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.709729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 944/1889] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:00.710697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.716713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 946/1890] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:00.721692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.725321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b08638-3702-4f80-a109-d6805ba3a83e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.730608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.731452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.731835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.732140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.732868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.734639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.748409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffbdc4-0dba-4886-8e2b-e018325e641a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.751668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 945/1891] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:00.753793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.757729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.758877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.759399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.759877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.761960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.764375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.766457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c6925-a9b8-4f9d-9800-7fa34b6564ea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.775159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.776288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.776720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.777145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] PATH_INFO: `/v3/projects/d53a92f329664e17b193d36e748318aa/users/5474feca504c4af9b65e6f8d148d1abc/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.777923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 947/1892] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:00.781949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.792973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1b754-bffa-4ab0-b0db-20c56a720f52 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.793333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:00.797455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53a92f329664e17b193d36e748318aa, user_id=5474feca504c4af9b65e6f8d148d1abc, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.798683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b698322-6572-40b9-837e-711b6970c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:00.799688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b698322-6572-40b9-837e-711b6970c343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.800151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b698322-6572-40b9-837e-711b6970c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.800321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b698322-6572-40b9-837e-711b6970c343 admin admin] PATH_INFO: `/v3/projects/4f3720266fb34530a618acf77a5c0aea/users/d803daf247044da6853043307b4a02fe/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.802771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.818076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b698322-6572-40b9-837e-711b6970c343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3720266fb34530a618acf77a5c0aea, user_id=d803daf247044da6853043307b4a02fe, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:00.821043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b698322-6572-40b9-837e-711b6970c343 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:00.824782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 946/1893] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:00 2022] PUT /identity/v3/projects/d53a92f329664e17b193d36e748318aa/users/5474feca504c4af9b65e6f8d148d1abc/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:00.828662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:00.848300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 948/1894] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:00 2022] PUT /identity/v3/projects/4f3720266fb34530a618acf77a5c0aea/users/d803daf247044da6853043307b4a02fe/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:00.859268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:00.859662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:00.860017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:00.889733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] MFA Rules not processed for user `d803daf247044da6853043307b4a02fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:00.890529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:00.906164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca24b66-f2a6-4136-9ed8-485336ff5bc7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:00.941027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:00.973128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 949/1895] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:01.004336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.005610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.006071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.006475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.007325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.009432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.017698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.020450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.027609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 947/1896] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:01.028670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.042669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4716f09-1482-488a-b14d-dfcec50cb240 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:01.048123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cef1c-65d0-4828-87d1-e6a9883ad585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.051191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cef1c-65d0-4828-87d1-e6a9883ad585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.051572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cef1c-65d0-4828-87d1-e6a9883ad585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.051880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cef1c-65d0-4828-87d1-e6a9883ad585 admin admin] PATH_INFO: `/v3/projects/d53a92f329664e17b193d36e748318aa/users/5474feca504c4af9b65e6f8d148d1abc/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.054801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cef1c-65d0-4828-87d1-e6a9883ad585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53a92f329664e17b193d36e748318aa, user_id=5474feca504c4af9b65e6f8d148d1abc, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.058514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cef1c-65d0-4828-87d1-e6a9883ad585 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.071879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 948/1897] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:01 2022] PUT /identity/v3/projects/d53a92f329664e17b193d36e748318aa/users/5474feca504c4af9b65e6f8d148d1abc/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:01.082271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9155237-ed3e-4da5-b42b-90757d1a3084 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:01.084921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.085219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.085650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.119269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] MFA Rules not processed for user `5474feca504c4af9b65e6f8d148d1abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:01.120748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.138366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.138983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.139288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.140012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.194013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 949/1898] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:01.201854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.203356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.205915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.207324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.213490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 950/1899] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:30:01.220661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bbee6-4330-41bf-be8a-55ed2cf0990b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:01.224285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfccc73-746a-46bf-9136-070aff5f209a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.224893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfccc73-746a-46bf-9136-070aff5f209a service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.225993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfccc73-746a-46bf-9136-070aff5f209a service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.226273 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfccc73-746a-46bf-9136-070aff5f209a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.229319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.232853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:01.238758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfccc73-746a-46bf-9136-070aff5f209a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.240754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfccc73-746a-46bf-9136-070aff5f209a service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.251312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 950/1900] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:30:01.285068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:01.296680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f17a-afe9-44a6-b0cc-00144d7e5992 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:01.302099 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.302757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.303031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.303299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] PATH_INFO: `/v3/users/d803daf247044da6853043307b4a02fe` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.304462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d803daf247044da6853043307b4a02fe)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.306605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.365318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd803daf247044da6853043307b4a02fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.381149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd803daf247044da6853043307b4a02fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.381842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd803daf247044da6853043307b4a02fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.382543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd803daf247044da6853043307b4a02fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.391744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd803daf247044da6853043307b4a02fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.423852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 951/1901] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:01 2022] DELETE /identity/v3/users/d803daf247044da6853043307b4a02fe => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:01.428378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.428661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.428922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.448875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:01.450081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.523563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 951/1902] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:01.530086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.531464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.598786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdfb79-0241-499e-88c7-f575a0483f1b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:01.602718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ba171-d877-4f04-8da4-03894f87a288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.603339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ba171-d877-4f04-8da4-03894f87a288 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.603648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ba171-d877-4f04-8da4-03894f87a288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.603923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ba171-d877-4f04-8da4-03894f87a288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.613586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ba171-d877-4f04-8da4-03894f87a288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.615066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ba171-d877-4f04-8da4-03894f87a288 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.625867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 952/1903] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:01.718706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.720241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.733406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82c2cf-ad02-4e1c-9d28-2590ed62415d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:01.738482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:01.739110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.739388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.739714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] PATH_INFO: `/v3/users/76dae40f0e834a2290a508f14472fad0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.740893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76dae40f0e834a2290a508f14472fad0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:01.743100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:01.829797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76dae40f0e834a2290a508f14472fad0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.838522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76dae40f0e834a2290a508f14472fad0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.839465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76dae40f0e834a2290a508f14472fad0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.849077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76dae40f0e834a2290a508f14472fad0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.866169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76dae40f0e834a2290a508f14472fad0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:01.872624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 952/1904] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:01 2022] DELETE /identity/v3/users/76dae40f0e834a2290a508f14472fad0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:01.877230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:01.877523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:01.877798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:01.897723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:01.899453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.964175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 953/1905] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:01.974263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.976314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:01.990430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:01.992672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.015787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4a3a9-c033-4c7d-806a-37fbcce4ce5d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.022645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.023303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.023599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.023874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] PATH_INFO: `/v3/projects/4f3720266fb34530a618acf77a5c0aea` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.025428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3720266fb34530a618acf77a5c0aea)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.028469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.048128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:02.067845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970161b-5693-4a66-baf7-f7f805e7a4d2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.071967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134845b-5298-4c38-b99f-32cb7b33d79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.072651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134845b-5298-4c38-b99f-32cb7b33d79e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.072989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134845b-5298-4c38-b99f-32cb7b33d79e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.073295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134845b-5298-4c38-b99f-32cb7b33d79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.084103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134845b-5298-4c38-b99f-32cb7b33d79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.085525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134845b-5298-4c38-b99f-32cb7b33d79e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.096999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3720266fb34530a618acf77a5c0aea', 'request_id': 'req-5fe057fb-efdb-40de-9316-9a133e38cdf3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.097959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 953/1906] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:02.103996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3720266fb34530a618acf77a5c0aea', 'request_id': 'req-5fe057fb-efdb-40de-9316-9a133e38cdf3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.113319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3720266fb34530a618acf77a5c0aea', 'request_id': 'req-5fe057fb-efdb-40de-9316-9a133e38cdf3'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.125426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] The token cache is being invalidate because project 4f3720266fb34530a618acf77a5c0aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:02.125720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.126393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5fe057fb-efdb-40de-9316-9a133e38cdf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.128886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 954/1907] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:01 2022] DELETE /identity/v3/projects/4f3720266fb34530a618acf77a5c0aea => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:02.133483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.133959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.134371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.163877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:02.165200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.172040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:02.257769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 954/1908] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:02.266546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.267603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.267603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.302109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:02.303954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.315612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 955/1909] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:02.317392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.328312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368252cc-dafb-4c45-9f93-d001e3aaf2f8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.333373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.334073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.334807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.334807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.335424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:02.339980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.342154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.358630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 955/1910] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:02.361122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.371761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607016ca-4fbd-4b42-9aeb-8dba0a9ddcb2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.377815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.378161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.378437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.378706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.379331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.381422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.406037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.408465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.416338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 956/1911] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:02 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:02.423091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.426035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246f11-6e8e-490e-9419-6207412b139a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.492239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.492901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.493168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.493436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] PATH_INFO: `/v3/projects/1b32e00055b44fd585972c3444160fb7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.494585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b32e00055b44fd585972c3444160fb7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.495991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a14bc-a2d7-40a8-871c-83ac5a461abd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.496593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.500751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.501379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.501658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.501951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.503434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.505434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.531991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 957/1912] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:02 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:02.533433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.547959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9b257-fd7e-48f9-9d2d-5fb3644b80e1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.568313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b32e00055b44fd585972c3444160fb7', 'request_id': 'req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.577667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b32e00055b44fd585972c3444160fb7', 'request_id': 'req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.584741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b32e00055b44fd585972c3444160fb7', 'request_id': 'req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.598509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] The token cache is being invalidate because project 1b32e00055b44fd585972c3444160fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:02.599412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.600621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.602696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 956/1913] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:02 2022] DELETE /identity/v3/projects/1b32e00055b44fd585972c3444160fb7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:02.606330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.607685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.626840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.628443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.628832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.629307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.630379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.633146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.635120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:02.653717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 958/1914] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:02.654804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.657437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.660943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:02.670893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f6dc9-9543-415e-9992-6d783be7f3f1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.676998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.677701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.678068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.678401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.679113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.680824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.697247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 959/1915] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:02.701555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.701982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87693ac0-48a1-4f7c-a995-4d3cffe0d6e9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.708134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.708802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.709083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.709352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] PATH_INFO: `/v3/users/5474feca504c4af9b65e6f8d148d1abc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.710500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=5474feca504c4af9b65e6f8d148d1abc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.712454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.714011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352518fd-d61f-4a92-8c94-7f460c05f5be admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.719351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ea146-8324-425b-814f-97e9112f436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.720227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea146-8324-425b-814f-97e9112f436d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.720903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea146-8324-425b-814f-97e9112f436d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.721155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea146-8324-425b-814f-97e9112f436d admin admin] PATH_INFO: `/v3/projects/5aa4b5762ee449c997838a883d12cc93/users/50681b2b349d4d389cee92a2eed4ed9e/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.741722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ea146-8324-425b-814f-97e9112f436d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa4b5762ee449c997838a883d12cc93, user_id=50681b2b349d4d389cee92a2eed4ed9e, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.744788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ea146-8324-425b-814f-97e9112f436d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:02.758850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 960/1916] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:02 2022] PUT /identity/v3/projects/5aa4b5762ee449c997838a883d12cc93/users/50681b2b349d4d389cee92a2eed4ed9e/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:02.770176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.770472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.770799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.784626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5474feca504c4af9b65e6f8d148d1abc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.794956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5474feca504c4af9b65e6f8d148d1abc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.795796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5474feca504c4af9b65e6f8d148d1abc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.805598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5474feca504c4af9b65e6f8d148d1abc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.809745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] MFA Rules not processed for user `50681b2b349d4d389cee92a2eed4ed9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:02.811155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.822041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5474feca504c4af9b65e6f8d148d1abc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:02.826081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 957/1917] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:02 2022] DELETE /identity/v3/users/5474feca504c4af9b65e6f8d148d1abc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:02.897517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 961/1918] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:02.898969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:02.900509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:02.985371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41be4f-d314-4ac6-a525-2967341af4ee admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:02.992005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:02.992865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:02.993319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:02.993749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:02.994544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:02.997472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.029484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 958/1919] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:02 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:03.033977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.104206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdb16f-a8b1-4ad5-84ef-1f539903e50d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.162199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:03.169581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.170204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.171575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.171575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.171892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.173860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.195786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.200254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:03.202844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 962/1920] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:03.206527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.214918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c9c1a-23f0-450a-a14c-3b5f3b041f4a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.218560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f5547-9ffd-40df-a948-36205f5129e9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.227137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.228008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.228171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.228587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.229444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.232914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.233565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.233844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.234205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] PATH_INFO: `/v3/projects/d53a92f329664e17b193d36e748318aa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.235379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53a92f329664e17b193d36e748318aa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.239940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.241698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.258912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 963/1921] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:03.260241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.273213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89399430-bd8f-4946-8fd4-e5cc1ff5b9be admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.278984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.279634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.280051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.280398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.281050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.282737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.291017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:03.301101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 964/1922] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:03.304924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.314505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53a92f329664e17b193d36e748318aa', 'request_id': 'req-145134ce-2674-40ee-af7d-6693ee54cfc1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:03.317743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedda66-cdf8-4a8d-90b4-57377949fdfd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.324307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53a92f329664e17b193d36e748318aa', 'request_id': 'req-145134ce-2674-40ee-af7d-6693ee54cfc1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:03.331884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53a92f329664e17b193d36e748318aa', 'request_id': 'req-145134ce-2674-40ee-af7d-6693ee54cfc1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:03.344451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] The token cache is being invalidate because project d53a92f329664e17b193d36e748318aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:03.344740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:03.345425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-145134ce-2674-40ee-af7d-6693ee54cfc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:03.347046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 959/1923] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:03 2022] DELETE /identity/v3/projects/d53a92f329664e17b193d36e748318aa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:03.352420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdd830-2370-4cd8-b537-22c29cfe45fb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.352713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdd830-2370-4cd8-b537-22c29cfe45fb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.352989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdd830-2370-4cd8-b537-22c29cfe45fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.379798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-17cdd830-2370-4cd8-b537-22c29cfe45fb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:03.381044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-17cdd830-2370-4cd8-b537-22c29cfe45fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:03.391048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e61ad4-1758-49b6-a3e9-9c3052304857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.391871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e61ad4-1758-49b6-a3e9-9c3052304857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.392160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e61ad4-1758-49b6-a3e9-9c3052304857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.392465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e61ad4-1758-49b6-a3e9-9c3052304857 admin admin] PATH_INFO: `/v3/projects/08da3f28dcd3479485665d6dbdd2d4e7/users/d245415dfb7441eba2ef440a7c19561a/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.410981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e61ad4-1758-49b6-a3e9-9c3052304857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08da3f28dcd3479485665d6dbdd2d4e7, user_id=d245415dfb7441eba2ef440a7c19561a, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.413823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e61ad4-1758-49b6-a3e9-9c3052304857 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.431530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 965/1924] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:03 2022] PUT /identity/v3/projects/08da3f28dcd3479485665d6dbdd2d4e7/users/d245415dfb7441eba2ef440a7c19561a/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:03.443246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.443588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.443839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.476558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 960/1925] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:03.484687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.485093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.485375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.488928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] MFA Rules not processed for user `d245415dfb7441eba2ef440a7c19561a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:03.490113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:03.511642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:03.513290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:03.566894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 966/1926] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:03.571986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.573811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:03.615969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 961/1927] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:03.618458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.631628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38d783-4595-43a3-ac5b-daf6e79fe23c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.633932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:03.638222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.638848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.639719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.640137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.640834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:03.641127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.642749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.664021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 962/1928] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:03.664194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611965-bf9b-48f3-947a-0b5bab3ea21d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.665307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.668126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.669475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.669951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.670787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.672470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.676481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.677364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee2f2b-fa39-4046-87e3-4cf5f6392738 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.682508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.683100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.683430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.683739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.684448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.686328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.712249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 967/1929] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:03.716134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.732344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 963/1930] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:03.736611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.798423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:03.857891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86747d91-4003-4910-adef-6dc0a318c943 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.875469 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a7080-db7d-43bf-9489-d90ebed0d1c4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.889865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7127e-fd82-4e0c-aee4-4832c8719a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.890507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7127e-fd82-4e0c-aee4-4832c8719a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.890783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7127e-fd82-4e0c-aee4-4832c8719a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.891056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7127e-fd82-4e0c-aee4-4832c8719a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.892260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7127e-fd82-4e0c-aee4-4832c8719a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.894121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7127e-fd82-4e0c-aee4-4832c8719a97 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.937414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 964/1931] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:03.940973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775aeb-5154-4324-801d-f076ac675f74 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.942100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775aeb-5154-4324-801d-f076ac675f74 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.942360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775aeb-5154-4324-801d-f076ac675f74 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.943937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 965/1932] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:30:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:30:03.944540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775aeb-5154-4324-801d-f076ac675f74 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:03.968847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775aeb-5154-4324-801d-f076ac675f74 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:03.973966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.974594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.975236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.975687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.976327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.978353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:03.984605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:03.985042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:03.985325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:03.985602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:03.988260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:03.992322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.007752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 968/1933] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:03 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:04.012951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.026172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 966/1934] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:04.027376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9737f-f146-4c27-9114-40d6e3214cdf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.028110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.032418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786b03c-b979-4cba-a6f2-3bf7a89c0c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.033332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786b03c-b979-4cba-a6f2-3bf7a89c0c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.033738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786b03c-b979-4cba-a6f2-3bf7a89c0c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.034023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786b03c-b979-4cba-a6f2-3bf7a89c0c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.035905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786b03c-b979-4cba-a6f2-3bf7a89c0c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.037807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786b03c-b979-4cba-a6f2-3bf7a89c0c9d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.044028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a1fec-1b6e-45f6-ae8e-0f7efdcffaef admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.053574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 969/1935] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:04.055223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.055515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.055814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.060535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.061303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.061587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.062695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.063526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.068793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.088968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:04.090293 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:04.092248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 967/1936] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:04.095096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.113989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cc174-2285-401e-b82a-6c67ca8892d2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.121770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.122835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.123225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.123579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] PATH_INFO: `/v3/projects/b22d696817c64739ab90896796ef068f/users/c023c9d1b39f449d85d9cb6a274c5847/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.153855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22d696817c64739ab90896796ef068f, user_id=c023c9d1b39f449d85d9cb6a274c5847, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.156864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.164702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:04.177480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 968/1937] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:04 2022] PUT /identity/v3/projects/b22d696817c64739ab90896796ef068f/users/c023c9d1b39f449d85d9cb6a274c5847/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:04.178852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.211299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 970/1938] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:30:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:04.213160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.228107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de488736-3728-4d6d-b132-832f05caf8d7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.328014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.328741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.329084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.329480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.329791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9298-bffe-472b-b1f9-760937554988 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.331028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.334974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.340837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b484-d567-48c5-834c-9c424aad4d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.341195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b484-d567-48c5-834c-9c424aad4d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.341568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b484-d567-48c5-834c-9c424aad4d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.341854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b484-d567-48c5-834c-9c424aad4d60 admin admin] PATH_INFO: `/v3/projects/f4fefcdea7824871add2e11a664ba682/users/36832cd9326145f5a0ba7f7c32517fe1/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.357975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 971/1939] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:04.362255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.362404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b484-d567-48c5-834c-9c424aad4d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fefcdea7824871add2e11a664ba682, user_id=36832cd9326145f5a0ba7f7c32517fe1, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.364980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b484-d567-48c5-834c-9c424aad4d60 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.377961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092054fe-e188-4e56-9912-31a5fcaa68ac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.383121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 969/1940] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:04 2022] PUT /identity/v3/projects/f4fefcdea7824871add2e11a664ba682/users/36832cd9326145f5a0ba7f7c32517fe1/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:04.395111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875854-5737-496e-8bf7-918480700b50 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.395867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875854-5737-496e-8bf7-918480700b50 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.396317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875854-5737-496e-8bf7-918480700b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.463766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-73875854-5737-496e-8bf7-918480700b50 None None] MFA Rules not processed for user `36832cd9326145f5a0ba7f7c32517fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:04.464998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-73875854-5737-496e-8bf7-918480700b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:04.489510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.491348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.491649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.491928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] PATH_INFO: `/v3/projects/b22d696817c64739ab90896796ef068f/users/c023c9d1b39f449d85d9cb6a274c5847/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.497512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22d696817c64739ab90896796ef068f, user_id=c023c9d1b39f449d85d9cb6a274c5847, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.504337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.542467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 972/1941] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:04 2022] PUT /identity/v3/projects/b22d696817c64739ab90896796ef068f/users/c023c9d1b39f449d85d9cb6a274c5847/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:04.542467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.542467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:04.554655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:04.611569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 970/1942] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:04.614011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.614435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.615202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.682414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79114c-9014-4638-b67f-a0bd9b9ad8fe admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.691416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] MFA Rules not processed for user `c023c9d1b39f449d85d9cb6a274c5847`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:04.693778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:04.703978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.706221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.706833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.707762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.716330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:04.719537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.722121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.738063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 973/1943] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:04.739355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.823629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7a565-2dc0-42fc-8251-ff7632aec2bb service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.827797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 971/1944] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:04.829226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.829934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.830302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.830684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.831688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.840683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.842337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:04.845387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:04.889683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 974/1945] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:04 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:04.894460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:04.952652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6b12a-658f-4ef7-9fd5-ee9d55e9421e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.972761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014ad81-aaf8-4332-b7f3-1eb6e61f7f02 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:04.977770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:04.978391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:04.978678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:04.978949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:04.979618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:04.981753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.010243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 975/1946] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:04 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:05.012502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:05.024315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.024558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ed566-fa3d-4ebd-84e2-f5243fa4c632 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:05.025146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.025495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.025843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.027737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:05.031686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.032501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.032856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.033516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.034544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.036940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.049207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 976/1947] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:05.053237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:05.062512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:05.063671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092b0f-6819-48b1-b5a2-673ff023dcd3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:05.069515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.070319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.070664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.071009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] PATH_INFO: `/v3/projects/e1d903bd75b3461ebdb50c7872ac54a3/users/7cf90f0e0cf64d89a4573ecbfb771090/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.087190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d903bd75b3461ebdb50c7872ac54a3, user_id=7cf90f0e0cf64d89a4573ecbfb771090, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.090019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.095027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.097391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.106258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 977/1948] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:05 2022] PUT /identity/v3/projects/e1d903bd75b3461ebdb50c7872ac54a3/users/7cf90f0e0cf64d89a4573ecbfb771090/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:05.108545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 972/1949] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:30:05.110259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:05.161881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:05.190631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f27bd6-4b7d-4257-af7f-6123a0372843 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:05.197703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819a11e-bc89-48f5-913a-77ac5eba5ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.199006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819a11e-bc89-48f5-913a-77ac5eba5ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.199285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819a11e-bc89-48f5-913a-77ac5eba5ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.199668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819a11e-bc89-48f5-913a-77ac5eba5ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.200364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819a11e-bc89-48f5-913a-77ac5eba5ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.201993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819a11e-bc89-48f5-913a-77ac5eba5ba6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.215630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 978/1950] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:05.218197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:05.231776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f47062-6861-43de-9e22-5e9d5b23dae7 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:05.237763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.238719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.239139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.239521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] PATH_INFO: `/v3/projects/e1d903bd75b3461ebdb50c7872ac54a3/users/7cf90f0e0cf64d89a4573ecbfb771090/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.241589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d903bd75b3461ebdb50c7872ac54a3, user_id=7cf90f0e0cf64d89a4573ecbfb771090, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.244565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.264862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 973/1951] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:05 2022] PUT /identity/v3/projects/e1d903bd75b3461ebdb50c7872ac54a3/users/7cf90f0e0cf64d89a4573ecbfb771090/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:05.270285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.270574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.270869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.309466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] MFA Rules not processed for user `7cf90f0e0cf64d89a4573ecbfb771090`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:05.310807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:05.409016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 979/1952] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:05.417761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:05.420541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:05.543262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f16e81-1186-432d-8e44-938233ef3b45 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:05.547210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.548151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.548526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.548909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.561405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.563574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.578893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 974/1953] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:05.694394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:05.710128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d905c6-0f6c-4193-9784-e4e6590539ed None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:05.714964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:05.715726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:05.716067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:05.716424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:05.822374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:05.824015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:05.838027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 980/1954] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:06.761311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:06.772422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90fedb-d647-4723-bdef-0373d7d06a22 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:06.776539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cde9f0-531a-4518-ab1d-a0b29a5f18b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:06.777421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cde9f0-531a-4518-ab1d-a0b29a5f18b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:06.777703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cde9f0-531a-4518-ab1d-a0b29a5f18b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:06.777973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cde9f0-531a-4518-ab1d-a0b29a5f18b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:06.889877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cde9f0-531a-4518-ab1d-a0b29a5f18b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:06.891007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cde9f0-531a-4518-ab1d-a0b29a5f18b3 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:06.923410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 975/1955] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:07.107160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:07.108708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:07.234501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:07.249848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17c852-6a2f-4ba3-8b14-193c6151f7dd service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:07.262836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:07.263497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:07.263803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:07.264078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] PATH_INFO: `/v3/users/0e77e728f7c84d09917246ca5efcf6fd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:07.265975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e77e728f7c84d09917246ca5efcf6fd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:07.268492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:07.382062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e77e728f7c84d09917246ca5efcf6fd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:07.399651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e77e728f7c84d09917246ca5efcf6fd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:07.401169 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e77e728f7c84d09917246ca5efcf6fd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:07.402828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e77e728f7c84d09917246ca5efcf6fd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:07.412879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e77e728f7c84d09917246ca5efcf6fd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:07.438278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 981/1956] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:07 2022] DELETE /identity/v3/users/0e77e728f7c84d09917246ca5efcf6fd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:07.444164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:07.444430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:07.444739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:07.471437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:07.472866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:07.555143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:07.567501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 976/1957] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:07.578044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:07.579791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:07.698984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256d69-78d1-4246-be89-7162f2c65e8f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:07.721654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:07.722720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:07.723331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:07.723649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:07.737145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:07.738752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:07.753296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 982/1958] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:07.845813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:07.847338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:07.943359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457248c6-2d7b-40e9-85b1-c9eca53e2af0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:07.950056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:07.950800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:07.951180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:07.951774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:07.954021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:08.043331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:08.057490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:08.134241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 977/1959] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:08.277639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:08.286653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:08.328723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d34d76-f208-40b5-86a8-8bf15c7c3c78 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:08.347884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:08.352108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:08.353664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:08.354191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] PATH_INFO: `/v3/projects/cb6b2759b80f4cfa93052c4cac85d957` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:08.356811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6b2759b80f4cfa93052c4cac85d957)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:08.359058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:08.502171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6b2759b80f4cfa93052c4cac85d957', 'request_id': 'req-3214282e-5c38-4430-befe-cc4a1c16c7a4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.514419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6b2759b80f4cfa93052c4cac85d957', 'request_id': 'req-3214282e-5c38-4430-befe-cc4a1c16c7a4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.526198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6b2759b80f4cfa93052c4cac85d957', 'request_id': 'req-3214282e-5c38-4430-befe-cc4a1c16c7a4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.550484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:08.560205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] The token cache is being invalidate because project cb6b2759b80f4cfa93052c4cac85d957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:08.560906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.561678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.568600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 983/1960] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:08 2022] DELETE /identity/v3/projects/cb6b2759b80f4cfa93052c4cac85d957 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:08.571725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:08.615658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d0621-f6c9-450d-9acb-c8839ce4f518 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:08.757433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:08.758456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:08.758957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:08.759435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] PATH_INFO: `/v3/users/ddb9df0aed9e456e940453aab684717e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:08.761156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb9df0aed9e456e940453aab684717e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:08.763530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:08.829314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb9df0aed9e456e940453aab684717e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.834132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:08.843001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb9df0aed9e456e940453aab684717e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.844556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb9df0aed9e456e940453aab684717e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.859321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb9df0aed9e456e940453aab684717e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.879144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c93abd9-b79d-409f-bb90-94804c204dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb9df0aed9e456e940453aab684717e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:08.887277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 978/1961] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:08 2022] DELETE /identity/v3/users/ddb9df0aed9e456e940453aab684717e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:09.269648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:09.271298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:09.296398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214282e-5c38-4430-befe-cc4a1c16c7a4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:09.464303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:09.464970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:09.465257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:09.465533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] PATH_INFO: `/v3/projects/2d00cd83b768420694904bce12b8f5aa` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:09.467984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d00cd83b768420694904bce12b8f5aa)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:09.470581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:09.604614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d00cd83b768420694904bce12b8f5aa', 'request_id': 'req-36075d42-81e1-418d-af1d-c1b17e84bd24'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:09.620349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d00cd83b768420694904bce12b8f5aa', 'request_id': 'req-36075d42-81e1-418d-af1d-c1b17e84bd24'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:09.631603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d00cd83b768420694904bce12b8f5aa', 'request_id': 'req-36075d42-81e1-418d-af1d-c1b17e84bd24'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:09.650029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] The token cache is being invalidate because project 2d00cd83b768420694904bce12b8f5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:09.650337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:09.651316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-36075d42-81e1-418d-af1d-c1b17e84bd24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:09.653387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 984/1962] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:09 2022] DELETE /identity/v3/projects/2d00cd83b768420694904bce12b8f5aa => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:09.659052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:09.659491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:09.659931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:09.687362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:09.688849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:09.770404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 979/1963] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:09.778867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:09.779218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:09.779503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:09.804884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:09.806235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:09.835602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 985/1964] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:09.838579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:09.853975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fd6de-babc-4870-bd81-a1b6b28f853b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:09.862033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:09.864247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:09.864533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:09.864804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:09.867784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:09.868437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:09.872093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:09.893291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 980/1965] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:09.908864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:09.918779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e835ffe-2ae1-4aec-aca7-55d17b8bbbfe None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:09.931828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:09.933946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:09.934245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:09.934507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:09.936050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:09.938449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:09.978457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 986/1966] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:09 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:09.983132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:09.996427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9de08-0f8d-4848-855b-5329aef3730f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.060984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.061613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.061899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.062176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.063597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.066091 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.098900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 981/1967] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:09 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:10.103379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.118060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a798ab9-5751-4504-885e-39786033769b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.123328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.124085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.124497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.124777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.125887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.127825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.139694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 987/1968] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:10.143734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.153963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc66729-a6f5-40e3-9b76-4df2b68840fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.158818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8955d-8cb4-4f89-9c2f-dfc17d7bee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.159630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8955d-8cb4-4f89-9c2f-dfc17d7bee36 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.160009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8955d-8cb4-4f89-9c2f-dfc17d7bee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.160358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8955d-8cb4-4f89-9c2f-dfc17d7bee36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.161094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8955d-8cb4-4f89-9c2f-dfc17d7bee36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.162970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8955d-8cb4-4f89-9c2f-dfc17d7bee36 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.175077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 982/1969] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:10.179181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.190555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d84e03-7604-4ece-958f-cad9067c3fb8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.195628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.196294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.196577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.196852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] PATH_INFO: `/v3/projects/851f8ae3fe36412c80b9abd39f91bd74/users/d9b099a858f649ad94dddff2fb78d221/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.214979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=851f8ae3fe36412c80b9abd39f91bd74, user_id=d9b099a858f649ad94dddff2fb78d221, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.217692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.233624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 988/1970] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:10 2022] PUT /identity/v3/projects/851f8ae3fe36412c80b9abd39f91bd74/users/d9b099a858f649ad94dddff2fb78d221/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:10.244409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.244707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.244983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.273537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] MFA Rules not processed for user `d9b099a858f649ad94dddff2fb78d221`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:10.274763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:10.301280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:10.346009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 983/1971] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:10.350825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.385126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:10.421003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c58ed-dca1-4105-a9ff-8e4506448d43 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.426741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.427388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.427708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.428048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.428702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.430719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.461508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 989/1972] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:10 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:10.462906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.474340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b52ff0-e9fc-4ef5-b8bd-682eda9d23bc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.534349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.534877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.535158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.535430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.536138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.538173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.563350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 984/1973] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:10 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:10.574069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.585509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b21838-207d-4b79-927b-2dc2d6e1d517 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.590959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.591662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.592015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.592387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.593000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.594885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.614604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 990/1974] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:10.616439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.629126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93cbb-4782-44b3-b4fb-3f989e7c4882 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.634671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.635427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.635892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.636180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] PATH_INFO: `/v3/projects/bb918b96977e48769e2c67063a59e505/users/941259a9ca1748c5a06e14a4803bca42/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.651966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb918b96977e48769e2c67063a59e505, user_id=941259a9ca1748c5a06e14a4803bca42, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.654622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.669332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 985/1975] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:10 2022] PUT /identity/v3/projects/bb918b96977e48769e2c67063a59e505/users/941259a9ca1748c5a06e14a4803bca42/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:10.673142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.683921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7da50-625a-494e-90f0-c54c38bf5613 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.756082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58613a-347d-4c22-b2ba-7537998a0618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.757112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58613a-347d-4c22-b2ba-7537998a0618 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.757708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58613a-347d-4c22-b2ba-7537998a0618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.758227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58613a-347d-4c22-b2ba-7537998a0618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.759181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58613a-347d-4c22-b2ba-7537998a0618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.761107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58613a-347d-4c22-b2ba-7537998a0618 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.773411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 991/1976] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:10.777622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.791924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca561fe-8027-42d6-a04f-75551c9d8fd9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:10.796941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:10.797586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.797875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.798181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] PATH_INFO: `/v3/projects/bb918b96977e48769e2c67063a59e505/users/941259a9ca1748c5a06e14a4803bca42/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.800325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb918b96977e48769e2c67063a59e505, user_id=941259a9ca1748c5a06e14a4803bca42, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:10.803014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:10.813955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 986/1977] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:10 2022] PUT /identity/v3/projects/bb918b96977e48769e2c67063a59e505/users/941259a9ca1748c5a06e14a4803bca42/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:10.824588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:10.824866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:10.825143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:10.855384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] MFA Rules not processed for user `941259a9ca1748c5a06e14a4803bca42`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:10.856730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:10.933253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 992/1978] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:10.945496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:10.947366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:11.024794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145552a2-fed2-4c1d-9f4b-6ee95d20df0a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:11.041421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69778e58-6eec-4fe0-995a-c299ec9396e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:11.042476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69778e58-6eec-4fe0-995a-c299ec9396e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:11.042996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69778e58-6eec-4fe0-995a-c299ec9396e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:11.043276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69778e58-6eec-4fe0-995a-c299ec9396e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:11.086370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:11.135607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69778e58-6eec-4fe0-995a-c299ec9396e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:11.137493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69778e58-6eec-4fe0-995a-c299ec9396e9 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:11.147957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 987/1979] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:15.052944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:15.054809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:15.084890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:15.130575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e427f8-c1d0-45a1-9198-5eac041269bf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:15.135108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:15.135780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:15.136092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:15.136369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] PATH_INFO: `/v3/users/50681b2b349d4d389cee92a2eed4ed9e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:15.137544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50681b2b349d4d389cee92a2eed4ed9e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:15.139425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:15.191978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50681b2b349d4d389cee92a2eed4ed9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:15.200160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50681b2b349d4d389cee92a2eed4ed9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:15.201062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50681b2b349d4d389cee92a2eed4ed9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:15.201804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50681b2b349d4d389cee92a2eed4ed9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:15.209914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50681b2b349d4d389cee92a2eed4ed9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:15.229023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 993/1980] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:15 2022] DELETE /identity/v3/users/50681b2b349d4d389cee92a2eed4ed9e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:15.232837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:15.233933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:15.234217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:15.254846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:15.256081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:15.320174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 988/1981] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:15.330893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:15.332321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:15.394702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9145ff-bcf8-47e2-954a-96e53c2e017c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:15.398324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:15.398956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:15.399236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:15.399507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:15.409065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:15.410493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:15.419841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 994/1982] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:15.934220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:15.936359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:15.949240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594b462-cbc3-40dc-8907-69499e37d7d2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:15.955180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:15.956296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:15.956867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:15.957607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] PATH_INFO: `/v3/projects/5aa4b5762ee449c997838a883d12cc93` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:15.960425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aa4b5762ee449c997838a883d12cc93)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:15.963360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:16.033640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa4b5762ee449c997838a883d12cc93', 'request_id': 'req-4f91347c-bc66-445a-b50c-d011870cd1fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.042732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa4b5762ee449c997838a883d12cc93', 'request_id': 'req-4f91347c-bc66-445a-b50c-d011870cd1fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.058649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa4b5762ee449c997838a883d12cc93', 'request_id': 'req-4f91347c-bc66-445a-b50c-d011870cd1fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.070954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] The token cache is being invalidate because project 5aa4b5762ee449c997838a883d12cc93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:16.071236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.071904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.073379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 989/1983] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:15 2022] DELETE /identity/v3/projects/5aa4b5762ee449c997838a883d12cc93 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:16.077428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:16.087845 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8dafcd-548d-4321-8fdb-f51b0bd5e075 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:16.133238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:16.154486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:16.155279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:16.155596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:16.155927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] PATH_INFO: `/v3/users/d245415dfb7441eba2ef440a7c19561a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:16.157168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d245415dfb7441eba2ef440a7c19561a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:16.160241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:16.212066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd245415dfb7441eba2ef440a7c19561a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.220968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd245415dfb7441eba2ef440a7c19561a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.221637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd245415dfb7441eba2ef440a7c19561a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.222234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd245415dfb7441eba2ef440a7c19561a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.231224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd245415dfb7441eba2ef440a7c19561a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.249521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 995/1984] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:16 2022] DELETE /identity/v3/users/d245415dfb7441eba2ef440a7c19561a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:16.553566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:16.555031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:16.569509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91347c-bc66-445a-b50c-d011870cd1fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:16.622712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:16.629368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:16.631215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:16.634993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:16.636158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:16.636608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:16.636885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] PATH_INFO: `/v3/users/36832cd9326145f5a0ba7f7c32517fe1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:16.638028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=36832cd9326145f5a0ba7f7c32517fe1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:16.640017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:16.650871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849595a3-a948-4499-8240-a9c0616d467b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:16.665367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:16.666016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:16.666291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:16.666564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] PATH_INFO: `/v3/projects/08da3f28dcd3479485665d6dbdd2d4e7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:16.667746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08da3f28dcd3479485665d6dbdd2d4e7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:16.669651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:16.693822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36832cd9326145f5a0ba7f7c32517fe1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.702754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36832cd9326145f5a0ba7f7c32517fe1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.703447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36832cd9326145f5a0ba7f7c32517fe1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.712630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36832cd9326145f5a0ba7f7c32517fe1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.723081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08da3f28dcd3479485665d6dbdd2d4e7', 'request_id': 'req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.726111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43cd9b73-09b9-43d1-8ee7-d622e1aaa36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36832cd9326145f5a0ba7f7c32517fe1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.728879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08da3f28dcd3479485665d6dbdd2d4e7', 'request_id': 'req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.734078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 990/1985] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:16 2022] DELETE /identity/v3/users/36832cd9326145f5a0ba7f7c32517fe1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:16.739575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08da3f28dcd3479485665d6dbdd2d4e7', 'request_id': 'req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.739825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:16.740215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:16.740626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:16.752230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] The token cache is being invalidate because project 08da3f28dcd3479485665d6dbdd2d4e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:16.752543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.753219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.754691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 996/1986] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:16 2022] DELETE /identity/v3/projects/08da3f28dcd3479485665d6dbdd2d4e7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:16.758564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:16.759976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:16.762870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:16.764144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:16.814550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:16.834529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4754b4-e46f-4ff3-8710-c71e90ea78f1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:16.835812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 991/1987] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:16.844792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:16.846229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:16.846498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:16.846804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] PATH_INFO: `/v3/users/c023c9d1b39f449d85d9cb6a274c5847` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:16.847241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:16.848287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c023c9d1b39f449d85d9cb6a274c5847)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:16.848659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:16.852677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:16.914115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c023c9d1b39f449d85d9cb6a274c5847'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.924639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c023c9d1b39f449d85d9cb6a274c5847'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.925369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c023c9d1b39f449d85d9cb6a274c5847'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.925997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c023c9d1b39f449d85d9cb6a274c5847'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.930507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abb00b-8a74-45e8-9a0e-8e540ed05d70 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:16.935316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:16.937073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:16.937585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:16.938007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:16.940706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:16.947088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c023c9d1b39f449d85d9cb6a274c5847'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:16.962004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:16.965782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:16.987787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 992/1988] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:16.991707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 997/1989] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:16 2022] DELETE /identity/v3/users/c023c9d1b39f449d85d9cb6a274c5847 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:17.067650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:17.069122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:17.093202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:17.162896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7bc87-09d3-4b8c-88b3-81f7793f498a service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:17.272068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:17.272740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:17.273023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:17.273334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:17.275214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:17.367859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:17.369989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:17.400461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 993/1990] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:17.562104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:17.571778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:17.572530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:17.575497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:17.697649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be13a30-f623-40da-bf0d-f8139e8eb1d7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:17.703726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:17.706514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:17.707132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:17.707487 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] PATH_INFO: `/v3/projects/f4fefcdea7824871add2e11a664ba682` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:17.709186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fefcdea7824871add2e11a664ba682)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:17.713359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:17.720353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a3b17-901a-4f14-92a9-e6da60d0fb38 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:17.726963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:17.728015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:17.728619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:17.729052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] PATH_INFO: `/v3/projects/b22d696817c64739ab90896796ef068f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:17.730591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b22d696817c64739ab90896796ef068f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:17.733378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:17.813440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:17.837319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b22d696817c64739ab90896796ef068f', 'request_id': 'req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.842314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fefcdea7824871add2e11a664ba682', 'request_id': 'req-7158e0cf-8e25-4f26-855f-8d0678beea94'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.850850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b22d696817c64739ab90896796ef068f', 'request_id': 'req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.859918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b22d696817c64739ab90896796ef068f', 'request_id': 'req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.865236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fefcdea7824871add2e11a664ba682', 'request_id': 'req-7158e0cf-8e25-4f26-855f-8d0678beea94'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.879447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] The token cache is being invalidate because project b22d696817c64739ab90896796ef068f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:17.880095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.880894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-994cd0d9-ff80-46e7-9a56-41efb7cd8c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.883601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 994/1991] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:17 2022] DELETE /identity/v3/projects/b22d696817c64739ab90896796ef068f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:17.890271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fefcdea7824871add2e11a664ba682', 'request_id': 'req-7158e0cf-8e25-4f26-855f-8d0678beea94'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.890819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0bb15-6bc3-4691-bbfb-86cf02a6ab0f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:17.891336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0bb15-6bc3-4691-bbfb-86cf02a6ab0f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:17.891884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0bb15-6bc3-4691-bbfb-86cf02a6ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:17.897401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:17.927888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] The token cache is being invalidate because project f4fefcdea7824871add2e11a664ba682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:17.927888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.927888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:17.935264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 998/1992] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:17 2022] DELETE /identity/v3/projects/f4fefcdea7824871add2e11a664ba682 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:17.940272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:17.942475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:17.956341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-49c0bb15-6bc3-4691-bbfb-86cf02a6ab0f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:17.957685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-49c0bb15-6bc3-4691-bbfb-86cf02a6ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.057600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158e0cf-8e25-4f26-855f-8d0678beea94 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.061360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 995/1993] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:18.069797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.070143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.070240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.070439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.070535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.070793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.071027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] PATH_INFO: `/v3/users/7cf90f0e0cf64d89a4573ecbfb771090` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.082188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf90f0e0cf64d89a4573ecbfb771090)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.083616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.107809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:18.111287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.138940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 996/1994] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:18.144620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.155757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de0c3-c369-46ce-8cb3-0214a87bb012 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.162987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf90f0e0cf64d89a4573ecbfb771090'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.164930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.165740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.166153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.166890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.167185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:18.167483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.169427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.178363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf90f0e0cf64d89a4573ecbfb771090'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.179574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf90f0e0cf64d89a4573ecbfb771090'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.181631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf90f0e0cf64d89a4573ecbfb771090'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.189860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 997/1995] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:30:18.190834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.192805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.197830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf90f0e0cf64d89a4573ecbfb771090'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.210464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcdb0e-cb40-4d85-bfea-b4ac17492426 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.217523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.218832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.219322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.219716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.221745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.224291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.237218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 999/1996] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:17 2022] DELETE /identity/v3/users/7cf90f0e0cf64d89a4573ecbfb771090 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:18.277435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 998/1997] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:18 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:18.281957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.404535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd30ab-dfce-4189-a6b6-30c0b594f41c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.443466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:18.517873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.518770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.518985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.519456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.520217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.522590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.552276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1000/1998] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:18 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:18.556305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.579507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a78f6-67cc-46f9-950e-f92cca976dba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.585157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.585899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.586274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.586660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.587411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.589249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.601593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 999/1999] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:18.603740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.606657 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.615277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.625501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864fc0-090d-4acc-b9a1-ac4e1c4da846 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.634429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b1932-8816-468c-adc9-5660b389df05 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.638912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.639733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.640021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.640473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] PATH_INFO: `/v3/projects/e1d903bd75b3461ebdb50c7872ac54a3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.641983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d903bd75b3461ebdb50c7872ac54a3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.644435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.646495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.647191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.647470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.647788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.649506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.651804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.668901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1000/2000] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:18.684700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.699299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e790-5992-447f-8ace-7477f0a6bcf7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:18.712243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d903bd75b3461ebdb50c7872ac54a3', 'request_id': 'req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.720735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d903bd75b3461ebdb50c7872ac54a3', 'request_id': 'req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.733400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d903bd75b3461ebdb50c7872ac54a3', 'request_id': 'req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.750993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] The token cache is being invalidate because project e1d903bd75b3461ebdb50c7872ac54a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:18.751356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.752215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9b268742-444e-42f1-a0f5-5d2ef8efcbe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:18.754095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1001/2001] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:18 2022] DELETE /identity/v3/projects/e1d903bd75b3461ebdb50c7872ac54a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:18.760402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565b6e-a7ef-4f78-ba60-819edb9d2b6c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.760757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565b6e-a7ef-4f78-ba60-819edb9d2b6c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.761106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565b6e-a7ef-4f78-ba60-819edb9d2b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.782326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:18.789833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-07565b6e-a7ef-4f78-ba60-819edb9d2b6c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:18.791450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-07565b6e-a7ef-4f78-ba60-819edb9d2b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.797784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69135b8e-d337-4012-8302-9f95534fc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:18.799584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69135b8e-d337-4012-8302-9f95534fc248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.800025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69135b8e-d337-4012-8302-9f95534fc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.800325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69135b8e-d337-4012-8302-9f95534fc248 admin admin] PATH_INFO: `/v3/projects/412bc6f61606415985b1fa3abe11975c/users/6f94b97783a548eba9f81da85e712913/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.802188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1002/2002] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:18.810794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb176-655b-4976-b248-14daf308120c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.811061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb176-655b-4976-b248-14daf308120c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.811353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb176-655b-4976-b248-14daf308120c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.826033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69135b8e-d337-4012-8302-9f95534fc248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412bc6f61606415985b1fa3abe11975c, user_id=6f94b97783a548eba9f81da85e712913, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:18.828770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69135b8e-d337-4012-8302-9f95534fc248 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:18.833597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d9deb176-655b-4976-b248-14daf308120c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:18.835116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d9deb176-655b-4976-b248-14daf308120c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.845492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1001/2003] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:18 2022] PUT /identity/v3/projects/412bc6f61606415985b1fa3abe11975c/users/6f94b97783a548eba9f81da85e712913/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:18.856816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:18.856816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:18.856816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:18.894171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] MFA Rules not processed for user `6f94b97783a548eba9f81da85e712913`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:18.895970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:18.946154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1003/2004] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:18.947144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb176-655b-4976-b248-14daf308120c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:18.978608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb176-655b-4976-b248-14daf308120c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.001109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.002132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.002619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.002856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.003645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:19.003903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.010739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.035382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1004/2005] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:19.043626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:19.092964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5243f3-0952-4e6d-a054-18c345986e27 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.115689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1002/2006] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:19.117799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.120859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.122590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.131041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.134092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.139099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:19.145086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:19.145242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.208986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:19.270942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1005/2007] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:19 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:19.275426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:19.306574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e711b-4e87-4563-a7cf-4bc4c1a34c51 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.457165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b1644-c927-408d-b767-aabbdb7f6db4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.462849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.463818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.464247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.464667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.466135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.468319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.502980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1006/2008] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:19 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:19.508123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:19.526197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc33b-865d-419d-80cf-7200b9237f65 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.528784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.532171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.532994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.533274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.544347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.545356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.547502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.548667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.549798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.552606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.568987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1007/2009] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:19.572512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:19.591827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2da44d-bf91-41d0-b3ad-cc98a8f2b048 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.599297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.600769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.601180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.601796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.602769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.609160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.625678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1008/2010] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:19.631711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:19.667626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12af43-6edf-4839-baf8-216d65c2d9c2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:19.679342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1f116-5a1a-4df1-9e2a-6e2bc59de4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:19.682119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1f116-5a1a-4df1-9e2a-6e2bc59de4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.682647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1f116-5a1a-4df1-9e2a-6e2bc59de4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.683084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1f116-5a1a-4df1-9e2a-6e2bc59de4ec admin admin] PATH_INFO: `/v3/projects/fb43eaa48ef6479ea8f5447fb5b7f96b/users/9e305f776b5d4ce4bede0fb0d6eaff6e/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.740538 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1f116-5a1a-4df1-9e2a-6e2bc59de4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb43eaa48ef6479ea8f5447fb5b7f96b, user_id=9e305f776b5d4ce4bede0fb0d6eaff6e, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.743635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1f116-5a1a-4df1-9e2a-6e2bc59de4ec admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.745101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:19.778692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1009/2011] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:19 2022] PUT /identity/v3/projects/fb43eaa48ef6479ea8f5447fb5b7f96b/users/9e305f776b5d4ce4bede0fb0d6eaff6e/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:19.790940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:19.793090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:19.794291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:19.840628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:19.846206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:19.876349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1003/2012] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:30:19.882185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] MFA Rules not processed for user `9e305f776b5d4ce4bede0fb0d6eaff6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:19.884768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:20.100648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1010/2013] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:20.105405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:20.122841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac787-eb2c-4dfa-b907-e4313e76ca52 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:20.411987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:20.416649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:20.419113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:20.422795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:20.431313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:20.447644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:20.623070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1004/2014] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:20 2022] POST /identity/v3/projects => generated 379 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:20.627315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:20.677597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b3780-42d7-4bda-b0c2-ebd2aeccc5d1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:20.832945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:20.915715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:20.919296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:20.919635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:20.919921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:20.922330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:20.925852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:20.958283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1011/2015] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:20 2022] POST /identity/v3/users => generated 296 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:20.958937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:20.980915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ff645-0a59-4b49-b9e8-db60a943d9d3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:21.004116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:21.004831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:21.005180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:21.005516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:21.006426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:21.008553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:21.051333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1005/2016] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:21.053434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:21.077578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af9d69-a50c-4fc4-b80d-7f2e6be23a50 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:21.089144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:21.089807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:21.090087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:21.090361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] PATH_INFO: `/v3/projects/384db24b39bc4569ac5f0406318fa2e3/users/dc79495f3d794cf689102e21ea28823e/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:21.113824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384db24b39bc4569ac5f0406318fa2e3, user_id=dc79495f3d794cf689102e21ea28823e, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:21.118024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:21.174383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1012/2017] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:21 2022] PUT /identity/v3/projects/384db24b39bc4569ac5f0406318fa2e3/users/dc79495f3d794cf689102e21ea28823e/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:21.178318 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:21.192990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e964f18-3174-4e9b-8060-f5916b09e0ae admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:21.300375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2ab39-7d50-4ef9-a13a-7f3fae17277d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:21.301451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2ab39-7d50-4ef9-a13a-7f3fae17277d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:21.301990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2ab39-7d50-4ef9-a13a-7f3fae17277d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:21.302569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2ab39-7d50-4ef9-a13a-7f3fae17277d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:21.303411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2ab39-7d50-4ef9-a13a-7f3fae17277d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:21.306406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2ab39-7d50-4ef9-a13a-7f3fae17277d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:21.322671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1006/2018] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:21.323890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:21.335056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16758626-f459-4faf-8305-ccbef19d92a4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:21.340011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:21.340681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:21.341515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:21.341515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] PATH_INFO: `/v3/projects/384db24b39bc4569ac5f0406318fa2e3/users/dc79495f3d794cf689102e21ea28823e/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:21.343111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384db24b39bc4569ac5f0406318fa2e3, user_id=dc79495f3d794cf689102e21ea28823e, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:21.346402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:21.359935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1013/2019] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:21 2022] PUT /identity/v3/projects/384db24b39bc4569ac5f0406318fa2e3/users/dc79495f3d794cf689102e21ea28823e/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:21.372018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:21.372315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:21.372597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:21.409123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] MFA Rules not processed for user `dc79495f3d794cf689102e21ea28823e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:21.410370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:21.473183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:21.532443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1007/2020] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:21.543292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:21.549541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:21.645229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620890-5369-4618-a203-3c40d5d04ba2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:21.651913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de1ef5-c673-4954-8272-32639452c331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:21.652453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de1ef5-c673-4954-8272-32639452c331 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:21.652734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de1ef5-c673-4954-8272-32639452c331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:21.653011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de1ef5-c673-4954-8272-32639452c331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:21.664424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de1ef5-c673-4954-8272-32639452c331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:21.666000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de1ef5-c673-4954-8272-32639452c331 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:21.687199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1014/2021] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:22.168483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:22.306680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a396b-9049-4e47-ac16-2d41be774de4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:22.315105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:22.316783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:22.317255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:22.318334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] PATH_INFO: `/v3/users/9e305f776b5d4ce4bede0fb0d6eaff6e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:22.321661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e305f776b5d4ce4bede0fb0d6eaff6e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:22.332064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:22.427252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e305f776b5d4ce4bede0fb0d6eaff6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:22.442829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e305f776b5d4ce4bede0fb0d6eaff6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:22.444315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e305f776b5d4ce4bede0fb0d6eaff6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:22.456025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e305f776b5d4ce4bede0fb0d6eaff6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:22.480816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e305f776b5d4ce4bede0fb0d6eaff6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:22.487374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1008/2022] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:22 2022] DELETE /identity/v3/users/9e305f776b5d4ce4bede0fb0d6eaff6e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:22.491654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:22.492024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:22.492307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:22.521071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:22.522142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:22.622879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1015/2023] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:22.636379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:22.637578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:22.713603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:22.718146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605dac-f0aa-4984-95aa-6f4b29420da7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:22.728520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:22.729375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:22.729751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:22.730068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:22.746440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:22.747972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:22.773137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1009/2024] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:23.200559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:23.202102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.223050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a227c-62c9-4772-88ab-b198f6304a45 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:23.228437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:23.229143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.229422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.230025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] PATH_INFO: `/v3/projects/fb43eaa48ef6479ea8f5447fb5b7f96b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.231255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb43eaa48ef6479ea8f5447fb5b7f96b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:23.233283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:23.259857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:23.280551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:23.282423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.302886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040913b1-fd4b-43c8-a4b5-a9f7f85ecf0b service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:23.311957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb43eaa48ef6479ea8f5447fb5b7f96b', 'request_id': 'req-42b35264-e327-4233-8b19-b5cb3a4e4aec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.320356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb43eaa48ef6479ea8f5447fb5b7f96b', 'request_id': 'req-42b35264-e327-4233-8b19-b5cb3a4e4aec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.332646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb43eaa48ef6479ea8f5447fb5b7f96b', 'request_id': 'req-42b35264-e327-4233-8b19-b5cb3a4e4aec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.353218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] The token cache is being invalidate because project fb43eaa48ef6479ea8f5447fb5b7f96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:23.353523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.354453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.359246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1016/2025] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:23 2022] DELETE /identity/v3/projects/fb43eaa48ef6479ea8f5447fb5b7f96b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:23.360363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:23.361809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.393421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:23.394394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.395323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.395807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] PATH_INFO: `/v3/users/d9b099a858f649ad94dddff2fb78d221` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.397421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b099a858f649ad94dddff2fb78d221)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:23.399752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:23.443004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35264-e327-4233-8b19-b5cb3a4e4aec admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:23.448375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:23.449022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.449308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.449584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] PATH_INFO: `/v3/users/dc79495f3d794cf689102e21ea28823e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.450731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc79495f3d794cf689102e21ea28823e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:23.452759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:23.457145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b099a858f649ad94dddff2fb78d221'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.466609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b099a858f649ad94dddff2fb78d221'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.467417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b099a858f649ad94dddff2fb78d221'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.478764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b099a858f649ad94dddff2fb78d221'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.496795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-911431b6-4e0a-4de4-a1f7-79fbf123a87a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b099a858f649ad94dddff2fb78d221'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.510614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1010/2026] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:23 2022] DELETE /identity/v3/users/d9b099a858f649ad94dddff2fb78d221 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:23.512573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.512864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.513138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.521220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc79495f3d794cf689102e21ea28823e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.530038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc79495f3d794cf689102e21ea28823e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.530947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc79495f3d794cf689102e21ea28823e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.532043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc79495f3d794cf689102e21ea28823e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.535723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:23.536944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.540427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc79495f3d794cf689102e21ea28823e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.561381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1017/2027] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:23 2022] DELETE /identity/v3/users/dc79495f3d794cf689102e21ea28823e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:23.566213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:23.568215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.603958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:23.627836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1011/2028] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:23.636127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:23.637585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.651943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:23.678122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1e5a1-153d-419e-9e6b-117be0a9b8b9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:23.689079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:23.690288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.690589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.690848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] PATH_INFO: `/v3/users/6f94b97783a548eba9f81da85e712913` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.692470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f94b97783a548eba9f81da85e712913)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:23.694957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:23.782070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f94b97783a548eba9f81da85e712913'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.786506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf3b27-79c8-4df2-a72f-7f56cd2a8b7e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:23.792947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f94b97783a548eba9f81da85e712913'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.794769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f94b97783a548eba9f81da85e712913'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.796052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f94b97783a548eba9f81da85e712913'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.812341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bdbab154-b32d-4abd-993b-cf9059a42ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f94b97783a548eba9f81da85e712913'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:23.870377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1018/2029] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:23 2022] DELETE /identity/v3/users/6f94b97783a548eba9f81da85e712913 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:23.875121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.875417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.876685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.899959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:23.901911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.921150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:23.922011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:23.924844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:23.927011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:23.933842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:23.959640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:23.962822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:23.975472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1012/2030] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:23.987235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:23.996446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.014250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1019/2031] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:24.020050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492714-6b08-4f58-9473-3c9ee057dcee service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.025370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.026953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.033487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.034157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.034443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.034714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] PATH_INFO: `/v3/projects/384db24b39bc4569ac5f0406318fa2e3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.039042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=384db24b39bc4569ac5f0406318fa2e3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.041019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.100656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:24.103232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384db24b39bc4569ac5f0406318fa2e3', 'request_id': 'req-dd03283e-ae3a-4631-969b-c6425eeb8b8c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.113029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384db24b39bc4569ac5f0406318fa2e3', 'request_id': 'req-dd03283e-ae3a-4631-969b-c6425eeb8b8c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.115277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0f46a-0dce-4f42-91b9-c6b344857423 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.125399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384db24b39bc4569ac5f0406318fa2e3', 'request_id': 'req-dd03283e-ae3a-4631-969b-c6425eeb8b8c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.138221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] The token cache is being invalidate because project 384db24b39bc4569ac5f0406318fa2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:24.138423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.139261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd03283e-ae3a-4631-969b-c6425eeb8b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.141262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1013/2032] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:23 2022] DELETE /identity/v3/projects/384db24b39bc4569ac5f0406318fa2e3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:24.146594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1eddc-c5a3-45ec-bcb7-1861987e5ac9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.146886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1eddc-c5a3-45ec-bcb7-1861987e5ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.147163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1eddc-c5a3-45ec-bcb7-1861987e5ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.174649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-77b1eddc-c5a3-45ec-bcb7-1861987e5ac9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:24.176033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-77b1eddc-c5a3-45ec-bcb7-1861987e5ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.184522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.185376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.185665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.185944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.187801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.231095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:24.247968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1014/2033] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:24.257111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82decde2-22da-4fc5-b05d-82788060490b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.257652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82decde2-22da-4fc5-b05d-82788060490b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.258187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82decde2-22da-4fc5-b05d-82788060490b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.262966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.264460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.276684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1020/2034] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:30:24.280366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-82decde2-22da-4fc5-b05d-82788060490b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:24.281681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82decde2-22da-4fc5-b05d-82788060490b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.289844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1015/2035] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:24.294057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.304204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6863-3cd9-4d79-aa4b-2cfc355e3919 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.309667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.310283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.310614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.310891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.311571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:24.311879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.314583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.330999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1021/2036] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:24.336204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82decde2-22da-4fc5-b05d-82788060490b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.343880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.347121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.348559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82decde2-22da-4fc5-b05d-82788060490b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.354244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.354863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.355147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.355425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.356149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.358058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.363718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c6f2-9795-4e87-8854-083c637d0d80 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.374807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.376044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.376827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.377102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] PATH_INFO: `/v3/projects/851f8ae3fe36412c80b9abd39f91bd74` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.379863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] RBAC: Authorizing `identity:delete_project(project_id=851f8ae3fe36412c80b9abd39f91bd74)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.384414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.404845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1016/2037] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:24.409212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.450705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851f8ae3fe36412c80b9abd39f91bd74', 'request_id': 'req-ef41d660-818e-4d40-95e6-f6590dc8111f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.460223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851f8ae3fe36412c80b9abd39f91bd74', 'request_id': 'req-ef41d660-818e-4d40-95e6-f6590dc8111f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.475276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851f8ae3fe36412c80b9abd39f91bd74', 'request_id': 'req-ef41d660-818e-4d40-95e6-f6590dc8111f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.484309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f44446-27a9-44ff-a41e-e1ef407999db admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.496825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] The token cache is being invalidate because project 851f8ae3fe36412c80b9abd39f91bd74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:24.498361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.499973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.502209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1022/2038] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:24 2022] DELETE /identity/v3/projects/851f8ae3fe36412c80b9abd39f91bd74 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:24.506775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.508633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.535157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:24.560655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.561315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.561598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.561877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.562685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.569885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.588362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1017/2039] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:24.592808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.595285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.595598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41d660-818e-4d40-95e6-f6590dc8111f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.600856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.601495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.601775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.602046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] PATH_INFO: `/v3/users/941259a9ca1748c5a06e14a4803bca42` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.603189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941259a9ca1748c5a06e14a4803bca42)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.605123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.609520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47dd2b-ee7a-4ea3-8cb7-4512e0d9a8ba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.615089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.615868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.616307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.616709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.617540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.619292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.634657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1018/2040] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:24.635862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.651598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a25ed-e82c-4656-8eb9-5bf2e145ed39 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.656631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.657798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.658096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.658532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.659390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.661040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.663014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941259a9ca1748c5a06e14a4803bca42'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.673081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1019/2041] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:24.674114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.675844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.677255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941259a9ca1748c5a06e14a4803bca42'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.677928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941259a9ca1748c5a06e14a4803bca42'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.678548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941259a9ca1748c5a06e14a4803bca42'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.686366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941259a9ca1748c5a06e14a4803bca42'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.689431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c28593-a849-4304-bccb-3af7c274b804 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.694197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.694815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.695101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.695375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] PATH_INFO: `/v3/projects/412bc6f61606415985b1fa3abe11975c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.696525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=412bc6f61606415985b1fa3abe11975c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.698502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.709424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1023/2042] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:24 2022] DELETE /identity/v3/users/941259a9ca1748c5a06e14a4803bca42 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:24.710673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:24.712204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.741374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:24.757732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412bc6f61606415985b1fa3abe11975c', 'request_id': 'req-43b69d37-90f7-4468-a279-e0e26a247575'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.768235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412bc6f61606415985b1fa3abe11975c', 'request_id': 'req-43b69d37-90f7-4468-a279-e0e26a247575'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.775639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412bc6f61606415985b1fa3abe11975c', 'request_id': 'req-43b69d37-90f7-4468-a279-e0e26a247575'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.790529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] The token cache is being invalidate because project 412bc6f61606415985b1fa3abe11975c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:24.791334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.791718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-43b69d37-90f7-4468-a279-e0e26a247575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:24.793505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1020/2043] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:24 2022] DELETE /identity/v3/projects/412bc6f61606415985b1fa3abe11975c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:24.798750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b7d2c-12aa-4942-9bc2-108fe7fd66f4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.799051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92382b47-03e1-4a26-afbe-12cb05f4b816 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:24.799856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b7d2c-12aa-4942-9bc2-108fe7fd66f4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.800208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b7d2c-12aa-4942-9bc2-108fe7fd66f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.827721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-811b7d2c-12aa-4942-9bc2-108fe7fd66f4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:24.829245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-811b7d2c-12aa-4942-9bc2-108fe7fd66f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.867237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:24.896354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8f08-0a73-4fcf-a174-9e84b0273c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:24.897111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8f08-0a73-4fcf-a174-9e84b0273c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.897573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8f08-0a73-4fcf-a174-9e84b0273c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.898050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8f08-0a73-4fcf-a174-9e84b0273c1b admin admin] PATH_INFO: `/v3/projects/4c77b54fca274cf6b85228920aa2097d/users/a9ce5437c2e74abd902aba0567afc4de/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.902848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1021/2044] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:24.912899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.912899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.912899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.921091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8f08-0a73-4fcf-a174-9e84b0273c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c77b54fca274cf6b85228920aa2097d, user_id=a9ce5437c2e74abd902aba0567afc4de, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:24.924010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8f08-0a73-4fcf-a174-9e84b0273c1b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:24.941278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:24.942664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:24.942799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1024/2045] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:24 2022] PUT /identity/v3/projects/4c77b54fca274cf6b85228920aa2097d/users/a9ce5437c2e74abd902aba0567afc4de/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:24.953371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:24.953668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:24.953949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:24.986160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] MFA Rules not processed for user `a9ce5437c2e74abd902aba0567afc4de`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:24.988024 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.047697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1022/2046] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:25.051435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.065108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93f8c5-c7fe-491b-8467-bd8b03f051ff None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.073537 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.074300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.074646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.074983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.075777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:25.076201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.078016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.092122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1023/2047] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:25.093104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.095248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.108660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648d291-3a1d-468d-ab41-9e5175d1ff61 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.114143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.114804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.115082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.115952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] PATH_INFO: `/v3/projects/bb918b96977e48769e2c67063a59e505` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.116520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb918b96977e48769e2c67063a59e505)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.118463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.120175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1025/2048] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:25.121203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.133903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb3125-f808-476d-a217-a6181d9ad9ed None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.139422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.140180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.140411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.140685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.141323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.143985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.174619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1026/2049] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:25.175484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.177429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.181081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:25.184932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb918b96977e48769e2c67063a59e505', 'request_id': 'req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:25.192653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb918b96977e48769e2c67063a59e505', 'request_id': 'req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:25.198298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb918b96977e48769e2c67063a59e505', 'request_id': 'req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:25.210710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] The token cache is being invalidate because project bb918b96977e48769e2c67063a59e505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:25.210992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:25.211697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:25.213205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1024/2050] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:25 2022] DELETE /identity/v3/projects/bb918b96977e48769e2c67063a59e505 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:25.213943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.215225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.246040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0553-e78e-4083-94f8-069fb80556bb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.250950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440fc10-a627-48b7-9785-58316fe84ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.251626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440fc10-a627-48b7-9785-58316fe84ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.251887 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440fc10-a627-48b7-9785-58316fe84ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.252159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440fc10-a627-48b7-9785-58316fe84ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.252953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440fc10-a627-48b7-9785-58316fe84ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.255730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440fc10-a627-48b7-9785-58316fe84ffe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.283441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1027/2051] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:25.284107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0414e-c70e-486b-b9e5-f6cb91c0ced7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.285039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.285327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.285606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.305050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:25.306248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.324252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:25.345180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.346035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.346343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.346648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.347294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.349293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.371911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1025/2052] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:25.375127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1028/2053] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:25.375924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.377537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.378917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.387037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f9bf9-3ff1-4539-9cd3-5a6a17bea8ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.390779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93680121-59ce-40ff-8a6f-0001f1c41351 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.391740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c682be3-cacf-45c8-8cad-e3947bd3b81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.392461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c682be3-cacf-45c8-8cad-e3947bd3b81e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.392747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c682be3-cacf-45c8-8cad-e3947bd3b81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.393021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c682be3-cacf-45c8-8cad-e3947bd3b81e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.393664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c682be3-cacf-45c8-8cad-e3947bd3b81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.395251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c682be3-cacf-45c8-8cad-e3947bd3b81e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.395995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.396648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.396949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.397223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.397878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.399812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.405358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1029/2054] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:25.406544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.406829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.407656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.427290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1026/2055] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/users => generated 291 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:25.427290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.431797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:25.433001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.435415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38576fd-050e-46ec-aca1-62e025bc54b0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.440322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.440864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1030/2056] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:25.441000 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.441180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.441457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.441661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.442164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.444186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.450962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc52ff-4a3a-4f2e-b7fb-91e736ba08bf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.455792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.455954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1027/2057] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:25.456398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.456675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.456793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.457008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.457587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.459183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.466928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b2404-e2b6-4c28-bab4-29b2a030062c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.472776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1031/2058] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:25.472776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.474996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.475653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.476132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.476400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.477073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:25.477371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.479049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.480927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01b90-6a16-48ce-b3ff-0eefd4be3719 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.485659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.486295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.486573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.486846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] PATH_INFO: `/v3/projects/09397aeb58f44f7c9f0ab921e471079a/users/472b6623f57a41568fe7ffce8e40e350/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.492312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1028/2059] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:25.496733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.502501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09397aeb58f44f7c9f0ab921e471079a, user_id=472b6623f57a41568fe7ffce8e40e350, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.505220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.507054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cd5f5-3a19-4155-969d-a03bb7c3b9cf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.512622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2daf0c-d63b-4653-9908-dbf1f630d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.513351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2daf0c-d63b-4653-9908-dbf1f630d6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.513698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2daf0c-d63b-4653-9908-dbf1f630d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.513967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2daf0c-d63b-4653-9908-dbf1f630d6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.514609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2daf0c-d63b-4653-9908-dbf1f630d6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.516206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2daf0c-d63b-4653-9908-dbf1f630d6ef admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.519127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1032/2060] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:25 2022] PUT /identity/v3/projects/09397aeb58f44f7c9f0ab921e471079a/users/472b6623f57a41568fe7ffce8e40e350/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:25.522792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.531804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1029/2061] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:25.533572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.534029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.534448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.561941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:25.563180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.578834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=4,keystone:SELECT=89,keystone:INSERT=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:25.590892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc793c-e06e-4617-9934-a2e7684cc056 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.595941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.596606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.596879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.597147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.597798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.599974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.632601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1033/2062] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:25.632601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.632906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1030/2063] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:25.633591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.644010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25db0-e216-4528-a7e0-247a190cd90a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.697892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:25.702640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.703262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.703375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d28e0-cc3e-44ba-9e51-749b0c300a55 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.703666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.703838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.704980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.707114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.709988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.710723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.711098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.711439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] PATH_INFO: `/v3/projects/4eb0db33d7f04eb5821646b60dda79da/users/d1c0b981b06343dfba7a7ee3b62a8e07/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.728734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb0db33d7f04eb5821646b60dda79da, user_id=d1c0b981b06343dfba7a7ee3b62a8e07, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.732577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1031/2064] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:25.732577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.732834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.732930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.746647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1034/2065] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:25 2022] PUT /identity/v3/projects/4eb0db33d7f04eb5821646b60dda79da/users/d1c0b981b06343dfba7a7ee3b62a8e07/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:25.747843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.796724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36c33b-a8bd-42f7-847e-5f3d95c09581 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.820334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bf4a8-8352-4ef9-95fe-b85e1153a7f7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.826239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bdd9c-d3f8-421d-85b4-2c29040f7408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.826889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bdd9c-d3f8-421d-85b4-2c29040f7408 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.827194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bdd9c-d3f8-421d-85b4-2c29040f7408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.827524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bdd9c-d3f8-421d-85b4-2c29040f7408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.828255 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bdd9c-d3f8-421d-85b4-2c29040f7408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.830596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bdd9c-d3f8-421d-85b4-2c29040f7408 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.842448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1035/2066] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:25.842978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.843283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.843574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.862810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.863432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.863764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.864042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.873885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.875227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.875729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] MFA Rules not processed for user `d1c0b981b06343dfba7a7ee3b62a8e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:25.877434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:25.885008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1032/2067] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:25.885918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.895875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a643a-f7f7-4f83-ab5f-f8581289b207 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.900574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.901174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.901459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.901732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.902405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.904054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.905881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:25.920704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1033/2068] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:25.921658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.931900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98227fb9-a4f8-4142-a4c3-7e763a575d40 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.942005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009a1b2-2143-47c5-abad-c4b92504fe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.942684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009a1b2-2143-47c5-abad-c4b92504fe0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.942958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009a1b2-2143-47c5-abad-c4b92504fe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.943230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009a1b2-2143-47c5-abad-c4b92504fe0b admin admin] PATH_INFO: `/v3/projects/f26fb14305d04acab9ed480c543d8adf/users/79a9f933480a4676a5f5acdb71b055cb/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.958727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1036/2069] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:25.963108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:25.969078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009a1b2-2143-47c5-abad-c4b92504fe0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26fb14305d04acab9ed480c543d8adf, user_id=79a9f933480a4676a5f5acdb71b055cb, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.974022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009a1b2-2143-47c5-abad-c4b92504fe0b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:25.986904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b654514-026e-4174-8c3a-99d16bc5f4d4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:25.994871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:25.996055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:25.996383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:25.996771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:25.998107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:25.999590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1034/2070] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:25 2022] PUT /identity/v3/projects/f26fb14305d04acab9ed480c543d8adf/users/79a9f933480a4676a5f5acdb71b055cb/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:26.000934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.019059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.019961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.020577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.038660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1037/2071] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:25 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:26.043224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.054210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] MFA Rules not processed for user `79a9f933480a4676a5f5acdb71b055cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:26.055572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.110641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:26.133288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76509431-1a0c-499b-a518-6df261be3ff5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.138732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.139350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.139712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.139955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.140589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.142481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1035/2072] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:26.144054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.153927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.155460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.185391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1038/2073] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:26 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:26.187268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.203724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323421da-91f9-46d1-9ea9-8b7045157963 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.210879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.212940 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.213816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.214393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.216463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.218334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.233073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c8680-b4c5-4f7a-b20e-d95aa81dd54d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.235280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1039/2074] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:26.237731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.238534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.238861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.239199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.239774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.248903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.250530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e12fb3-628c-4b14-a74a-75386c6cc316 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.251620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.256068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.256884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.257167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.257442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] PATH_INFO: `/v3/projects/b9bd540eb18b4726a0d3b229ec974dd1/users/f04a66b94d5847c5995a8f8b80c53ac6/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.265008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1036/2075] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:30:26.289008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.298244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bd540eb18b4726a0d3b229ec974dd1, user_id=f04a66b94d5847c5995a8f8b80c53ac6, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.303311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2aa54-b001-4dfd-a207-0375c4e41de9 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.304506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.309498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.310436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.310759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.311054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] PATH_INFO: `/v3/users/79a9f933480a4676a5f5acdb71b055cb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.312427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a9f933480a4676a5f5acdb71b055cb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.315260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.342094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1040/2076] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:26 2022] PUT /identity/v3/projects/b9bd540eb18b4726a0d3b229ec974dd1/users/f04a66b94d5847c5995a8f8b80c53ac6/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:26.344165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.387707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a9f933480a4676a5f5acdb71b055cb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:26.398459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a9f933480a4676a5f5acdb71b055cb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:26.399258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a9f933480a4676a5f5acdb71b055cb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:26.405490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:26.414007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a9f933480a4676a5f5acdb71b055cb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:26.430966 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-72a6524d-86f0-4bb0-a3b0-850fa6685c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a9f933480a4676a5f5acdb71b055cb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:26.438395 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1037/2077] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:26 2022] DELETE /identity/v3/users/79a9f933480a4676a5f5acdb71b055cb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:26.446360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.446653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.447178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.470220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:26.471662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.542453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b20eb-cc8e-46b8-a68f-c7fb06c41c81 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.582914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1038/2078] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:26.596706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.598491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.639638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.640432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.640711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.640986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.643164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.645010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.660182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1041/2079] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:26.663846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.665688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.676758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53267c6e-7602-4b2a-b4d2-7bd2e8690615 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.680704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6ead-4c9a-4f24-b223-ea07f6114347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.681337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6ead-4c9a-4f24-b223-ea07f6114347 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.681596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6ead-4c9a-4f24-b223-ea07f6114347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.681867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6ead-4c9a-4f24-b223-ea07f6114347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.682119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e4a2c-c958-4114-8f39-53f2a60098ab admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.686908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.687587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.687873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.688153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] PATH_INFO: `/v3/projects/b9bd540eb18b4726a0d3b229ec974dd1/users/f04a66b94d5847c5995a8f8b80c53ac6/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.691209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bd540eb18b4726a0d3b229ec974dd1, user_id=f04a66b94d5847c5995a8f8b80c53ac6, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.692095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6ead-4c9a-4f24-b223-ea07f6114347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:26.693523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6ead-4c9a-4f24-b223-ea07f6114347 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.694055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:26.705442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1039/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:26.706367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1042/2081] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:26 2022] PUT /identity/v3/projects/b9bd540eb18b4726a0d3b229ec974dd1/users/f04a66b94d5847c5995a8f8b80c53ac6/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:26.719048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.719448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.719904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.762685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] MFA Rules not processed for user `f04a66b94d5847c5995a8f8b80c53ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:26.764124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.856614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1040/2082] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:26.869838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:26.872596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:26.918482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:26.967035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2eed3-4f6e-4b95-ad55-0e473fd6679e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:26.973019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:26.974096 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:26.974484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:26.975142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:26.977419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:27.064595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:27.067486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b08a7-c4c9-486d-9cbf-b214f9fdff3f service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:27.080345 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1043/2083] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:27.157567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:27.158657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:27.169451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:27.320885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41eb2b-3d04-483f-b415-0dca08aeb7dd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:27.332228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:27.333094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:27.333386 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:27.335175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] PATH_INFO: `/v3/projects/f26fb14305d04acab9ed480c543d8adf` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:27.337224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26fb14305d04acab9ed480c543d8adf)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:27.342429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:27.460319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26fb14305d04acab9ed480c543d8adf', 'request_id': 'req-12614f6c-aa4e-4e65-b973-54fe3ee57a48'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:27.488155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26fb14305d04acab9ed480c543d8adf', 'request_id': 'req-12614f6c-aa4e-4e65-b973-54fe3ee57a48'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:27.505765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26fb14305d04acab9ed480c543d8adf', 'request_id': 'req-12614f6c-aa4e-4e65-b973-54fe3ee57a48'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:27.530310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] The token cache is being invalidate because project f26fb14305d04acab9ed480c543d8adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:27.531174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:27.533550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12614f6c-aa4e-4e65-b973-54fe3ee57a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:27.538479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1041/2084] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:27 2022] DELETE /identity/v3/projects/f26fb14305d04acab9ed480c543d8adf => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:27.549082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:27.549082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:27.549082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:27.576762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:27.578014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:27.702542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1044/2085] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:27.711148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:27.711467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:27.711872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:27.757914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:27.759733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:27.780358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:27.781694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1042/2086] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:27.793149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f760e0-8f68-4b39-95a6-14c52e66978d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:27.798821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:27.799720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:27.799853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:27.800017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:27.800727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:27.801484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:27.802969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:27.818144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1045/2087] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:27.821896 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:27.837893 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bcef4-adc9-4ea5-908f-a48945132361 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:27.847792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:27.848437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:27.848718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:27.849039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:27.849976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:27.852694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:27.914473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1043/2088] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:27 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:27.918546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:27.937413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded29e99-8ff3-4f4f-b428-4e4616b1b2ba admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:28.030729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:28.031433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:28.033236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:28.033524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:28.034922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:28.037688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:28.057726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:28.065637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1046/2089] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:27 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:28.070298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:28.085861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ba5d-3041-46bd-b8f1-b56b33b4a98f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:28.090842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:28.091614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:28.091969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:28.092317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:28.093071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:28.095686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:28.111129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1044/2090] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:28.112429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:28.125698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c705e-50bd-4178-a5a9-04fab232656e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:28.146030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd280433-80b2-4f8c-bc59-2060bd91fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:28.148184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd280433-80b2-4f8c-bc59-2060bd91fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:28.148184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd280433-80b2-4f8c-bc59-2060bd91fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:28.148184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd280433-80b2-4f8c-bc59-2060bd91fb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:28.148184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd280433-80b2-4f8c-bc59-2060bd91fb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:28.149686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd280433-80b2-4f8c-bc59-2060bd91fb8f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:28.167360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1047/2091] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:28.171165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:28.191720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a2da8-f925-4e5e-a1c7-3bd6f37f8ee0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:28.196539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:28.197213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:28.197497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:28.197778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] PATH_INFO: `/v3/projects/c5c53db0f2074b2b90513cce89fe9307/users/0f9d3fe911eb4688a3d59820202406d4/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:28.224433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c53db0f2074b2b90513cce89fe9307, user_id=0f9d3fe911eb4688a3d59820202406d4, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:28.227412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:28.259628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1045/2092] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:28 2022] PUT /identity/v3/projects/c5c53db0f2074b2b90513cce89fe9307/users/0f9d3fe911eb4688a3d59820202406d4/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:28.271005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:28.271385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:28.271806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:28.305414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] MFA Rules not processed for user `0f9d3fe911eb4688a3d59820202406d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:28.306685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:28.395453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1048/2093] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:28.406519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:28.409286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:28.445505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:28.474347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502808ab-4206-40f6-90db-7ea21f9b5f60 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:28.478550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:28.479295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:28.479688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:28.480031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:28.489973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:28.491414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:28.502613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1046/2094] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:29.074663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:29.098316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab19cb-ab89-4c33-a1ba-2e5290c15e4d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:29.102956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:29.103687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:29.104041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:29.104385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:29.107127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:29.189196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:29.191164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36befd5c-1901-400e-9fcc-bc2c24b5da9e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:29.204052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1049/2095] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:30.184140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:30.196829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f912ba-b7c6-478a-a41c-52a3cc675db8 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:30.202063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:30.202683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:30.202970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:30.203243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:30.206235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:30.295300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:30.297421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:30.318065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1047/2096] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:32.629324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:32.630219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:32.630498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:32.664018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] MFA Rules not processed for user `472b6623f57a41568fe7ffce8e40e350`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:32.665372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:32.733222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1050/2097] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:32.737977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:32.739992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:32.820145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c1a2-c279-4731-946b-8a56b13bf694 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:32.825575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:32.826410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:32.827017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:32.827455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:32.828144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:32.831175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:32.873846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1048/2098] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:32 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:32.875287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:32.883996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:32.896766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0d89-c736-4ffb-afec-498647a6172a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:32.984366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:32.985180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:32.985499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:32.985863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:32.986509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:32.988498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:33.014315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1051/2099] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:32 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:33.019133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:33.030044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59391-22f3-4cda-9115-9c1f7ccdb509 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:33.035196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:33.035868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:33.036150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:33.036421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:33.037085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:33.038872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:33.051554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1049/2100] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:33.056098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:33.067205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3394c9-f6c8-45d6-ac13-d0e950a4e67f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:33.072126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:33.072842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:33.073121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:33.073394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] PATH_INFO: `/v3/projects/81f3ef01e71946bb96dc617898d0d8b8/users/a773d2f22532427b9ca87d721c7798a0/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:33.091534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f3ef01e71946bb96dc617898d0d8b8, user_id=a773d2f22532427b9ca87d721c7798a0, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:33.094380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:33.114467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1052/2101] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:33 2022] PUT /identity/v3/projects/81f3ef01e71946bb96dc617898d0d8b8/users/a773d2f22532427b9ca87d721c7798a0/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:33.116350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:33.129366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d3985-099b-442f-a7fd-db7301ed654e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:33.213527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:33.236124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:33.236920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:33.237334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:33.237679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:33.238574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:33.240953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:33.255740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1050/2102] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:33.257119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:33.268506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c660c-4de3-4967-a265-040892575c16 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:33.275772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e4423-3242-4d49-aa82-b1a3add78441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:33.276707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e4423-3242-4d49-aa82-b1a3add78441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:33.277026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e4423-3242-4d49-aa82-b1a3add78441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:33.277355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e4423-3242-4d49-aa82-b1a3add78441 admin admin] PATH_INFO: `/v3/projects/81f3ef01e71946bb96dc617898d0d8b8/users/a773d2f22532427b9ca87d721c7798a0/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:33.279798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e4423-3242-4d49-aa82-b1a3add78441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f3ef01e71946bb96dc617898d0d8b8, user_id=a773d2f22532427b9ca87d721c7798a0, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:33.283330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e4423-3242-4d49-aa82-b1a3add78441 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:33.295815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1053/2103] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:33 2022] PUT /identity/v3/projects/81f3ef01e71946bb96dc617898d0d8b8/users/a773d2f22532427b9ca87d721c7798a0/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:34.709506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:34.713108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:34.739447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2414ea-2a9a-4466-a8ab-8aecff8c532f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:34.850266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:34.851818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:34.852182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:34.852541 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] PATH_INFO: `/v3/users/a9ce5437c2e74abd902aba0567afc4de` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:34.853910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ce5437c2e74abd902aba0567afc4de)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:34.857279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:34.945134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ce5437c2e74abd902aba0567afc4de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:34.960250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ce5437c2e74abd902aba0567afc4de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:34.961958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ce5437c2e74abd902aba0567afc4de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:34.979245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ce5437c2e74abd902aba0567afc4de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:34.999617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ce5437c2e74abd902aba0567afc4de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:35.022481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1051/2104] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:34 2022] DELETE /identity/v3/users/a9ce5437c2e74abd902aba0567afc4de => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:35.022877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:35.023377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:35.023471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:35.081175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:35.082437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:35.365031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1054/2105] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:35.413278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:35.416730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:35.670916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df554-0dc5-4a30-aa01-98009914d7b7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:35.688441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:35.688441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:35.689175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:35.689713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:35.717576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:35.720504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:35.760513 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:35.780127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1052/2106] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:37.220441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:37.228628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:37.266963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2c362-33fc-426b-9e0d-04c349f4dc37 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:37.287557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:37.288276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:37.288555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:37.288832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] PATH_INFO: `/v3/projects/4c77b54fca274cf6b85228920aa2097d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:37.296606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c77b54fca274cf6b85228920aa2097d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:37.299256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:37.441195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c77b54fca274cf6b85228920aa2097d', 'request_id': 'req-a857b342-e998-46b9-8098-0db4f0c59742'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.444454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:37.457547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c77b54fca274cf6b85228920aa2097d', 'request_id': 'req-a857b342-e998-46b9-8098-0db4f0c59742'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.477429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c77b54fca274cf6b85228920aa2097d', 'request_id': 'req-a857b342-e998-46b9-8098-0db4f0c59742'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.504731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] The token cache is being invalidate because project 4c77b54fca274cf6b85228920aa2097d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:37.505476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.506550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.508691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1055/2107] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:37 2022] DELETE /identity/v3/projects/4c77b54fca274cf6b85228920aa2097d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:37.513580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:37.540766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0b4cd-70c0-4dad-b156-0104c62fba97 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:37.785705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:37.789003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:37.789601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:37.790025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] PATH_INFO: `/v3/users/d1c0b981b06343dfba7a7ee3b62a8e07` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:37.792437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c0b981b06343dfba7a7ee3b62a8e07)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:37.798961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:37.932331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c0b981b06343dfba7a7ee3b62a8e07'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.947392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c0b981b06343dfba7a7ee3b62a8e07'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.948609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c0b981b06343dfba7a7ee3b62a8e07'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.962334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c0b981b06343dfba7a7ee3b62a8e07'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.983563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c0b981b06343dfba7a7ee3b62a8e07'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:37.998055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1053/2108] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:37 2022] DELETE /identity/v3/users/d1c0b981b06343dfba7a7ee3b62a8e07 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:39.434889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:39.440324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:39.463492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857b342-e998-46b9-8098-0db4f0c59742 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:39.678341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:39.680365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:39.681450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:39.685641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] PATH_INFO: `/v3/projects/4eb0db33d7f04eb5821646b60dda79da` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:39.689993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb0db33d7f04eb5821646b60dda79da)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:39.707809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:39.826726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb0db33d7f04eb5821646b60dda79da', 'request_id': 'req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:39.838781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb0db33d7f04eb5821646b60dda79da', 'request_id': 'req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:39.854950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb0db33d7f04eb5821646b60dda79da', 'request_id': 'req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:39.884190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] The token cache is being invalidate because project 4eb0db33d7f04eb5821646b60dda79da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:39.884480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:39.885555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:39.887383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1056/2109] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:39 2022] DELETE /identity/v3/projects/4eb0db33d7f04eb5821646b60dda79da => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:39.892349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:39.896211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:39.921385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7e5a-deec-4ef0-b1a6-c33c5b0cba3c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:40.082158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:40.084298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:40.084298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:40.087742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] PATH_INFO: `/v3/users/f04a66b94d5847c5995a8f8b80c53ac6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:40.089237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04a66b94d5847c5995a8f8b80c53ac6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:40.093070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:40.125401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:40.177996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04a66b94d5847c5995a8f8b80c53ac6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:40.193248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04a66b94d5847c5995a8f8b80c53ac6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:40.193738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04a66b94d5847c5995a8f8b80c53ac6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:40.207121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04a66b94d5847c5995a8f8b80c53ac6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:40.285133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a7cad21-3271-4bf0-b326-3f3d10442d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04a66b94d5847c5995a8f8b80c53ac6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:40.293736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1054/2110] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:39 2022] DELETE /identity/v3/users/f04a66b94d5847c5995a8f8b80c53ac6 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:40.968380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:40.979155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:41.026609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0911d6-06bb-4bd9-9de0-cb4aca1e85f1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:41.181002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:41.182092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.182592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.183082 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] PATH_INFO: `/v3/projects/b9bd540eb18b4726a0d3b229ec974dd1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.186145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bd540eb18b4726a0d3b229ec974dd1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:41.189073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:41.217048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:41.308456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bd540eb18b4726a0d3b229ec974dd1', 'request_id': 'req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.322965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bd540eb18b4726a0d3b229ec974dd1', 'request_id': 'req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.338459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bd540eb18b4726a0d3b229ec974dd1', 'request_id': 'req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.358691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] The token cache is being invalidate because project b9bd540eb18b4726a0d3b229ec974dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:41.359790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.360672 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-bc25b9f1-38d3-4a5c-8237-db037a0dbe9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.362402 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1057/2111] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:40 2022] DELETE /identity/v3/projects/b9bd540eb18b4726a0d3b229ec974dd1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:41.367801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.368622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.369054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.399304 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:41.400817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:41.527999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1055/2112] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:41.536428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.536848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.537246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.571851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:41.573546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:41.583705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:41.585778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:41.592609 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d80763-4ce6-4a12-85b4-1d7d8361f970 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:41.598552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:41.600161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.600942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.602042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] PATH_INFO: `/v3/users/0f9d3fe911eb4688a3d59820202406d4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.603968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9d3fe911eb4688a3d59820202406d4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:41.606213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:41.607669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1058/2113] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:41.610885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:41.633652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71e2c-335b-4b39-86b0-49903d1ab409 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:41.650935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:41.653907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.654266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.654728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.655572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:41.656570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:41.660673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:41.686069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1059/2114] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:41.690095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:41.708420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9d3fe911eb4688a3d59820202406d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.708703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cb7bb-d87a-4ca2-b61b-a40395a0d84d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:41.719382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad189f1-efee-4aee-afb7-f9e40e2b636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:41.720439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad189f1-efee-4aee-afb7-f9e40e2b636f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.721041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad189f1-efee-4aee-afb7-f9e40e2b636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.721704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad189f1-efee-4aee-afb7-f9e40e2b636f admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.723124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad189f1-efee-4aee-afb7-f9e40e2b636f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:41.726726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad189f1-efee-4aee-afb7-f9e40e2b636f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:41.730259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9d3fe911eb4688a3d59820202406d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.731581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9d3fe911eb4688a3d59820202406d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.753407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9d3fe911eb4688a3d59820202406d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.770274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1060/2115] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:41 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:41.773038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693766-6bd3-4020-abb4-7b74a6f7154e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.773386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693766-6bd3-4020-abb4-7b74a6f7154e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.773735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693766-6bd3-4020-abb4-7b74a6f7154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.787636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9d3fe911eb4688a3d59820202406d4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:41.808014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1056/2116] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:41 2022] DELETE /identity/v3/users/0f9d3fe911eb4688a3d59820202406d4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:41.811079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:41.814495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:41.848898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:41.859675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e3693766-6bd3-4020-abb4-7b74a6f7154e None None] MFA Rules not processed for user `a773d2f22532427b9ca87d721c7798a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:41.861509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e3693766-6bd3-4020-abb4-7b74a6f7154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:41.950248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1369f-b103-4b7a-af67-2eb7860888d2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:41.956861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:41.957572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.958084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.958429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:41.959143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:41.961100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:41.981409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1061/2117] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:41.984250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:41.985793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:41.986212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.003353 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1057/2118] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:41 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:42.006343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:42.011481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:42.012807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:42.022349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1062/2119] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:42.023770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:42.027431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:42.029918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad566d2-bf90-43b5-ae5f-b69702357b90 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:42.036608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:42.037779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.038424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.039594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.040458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:42.043144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:42.056519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1058/2120] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:42.058183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:42.060171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:42.104006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:42.162299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc6948-b9be-46df-840f-fc0b527d5fb4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:42.167786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7666a73-82ee-4ed9-b080-f34fe455c0a2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:42.168791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:42.171597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.171597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.171597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.178591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:42.180079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.180372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.181161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.189686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:42.192103 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:42.192536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:42.215464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1059/2121] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:42.216478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:42.233266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db33c-25e5-4499-b199-b66c38d79934 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:42.243447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:42.244097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.244379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.244653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.245312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:42.249447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:42.270003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1060/2122] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:42.275167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:42.298856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040dbb5-faa7-485a-a1eb-17eb50daa6bd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:42.313238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:42.313807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.314086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.314360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] PATH_INFO: `/v3/projects/6ceb527ce5a74fc98bb7ca6fcb0fabba/users/902e421fe49444318dace820aaa881f2/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.325324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:42.328013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab321e4-474b-4eb7-8625-24413073d53a service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:42.342432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1063/2123] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:42.353670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ceb527ce5a74fc98bb7ca6fcb0fabba, user_id=902e421fe49444318dace820aaa881f2, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:42.356763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:42.392371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1061/2124] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:42 2022] PUT /identity/v3/projects/6ceb527ce5a74fc98bb7ca6fcb0fabba/users/902e421fe49444318dace820aaa881f2/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:42.404447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.405147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.405629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.489065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] MFA Rules not processed for user `902e421fe49444318dace820aaa881f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:42.490689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:42.656296 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1064/2125] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:42.673017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:42.675715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:42.831901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a196c-d576-45a7-98af-3df13dbebc76 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:42.836576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623b597-15c7-4610-9f7f-dd7fba1cbf43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:42.837669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623b597-15c7-4610-9f7f-dd7fba1cbf43 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:42.838403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623b597-15c7-4610-9f7f-dd7fba1cbf43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:42.840299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623b597-15c7-4610-9f7f-dd7fba1cbf43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:42.861616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623b597-15c7-4610-9f7f-dd7fba1cbf43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:42.863090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623b597-15c7-4610-9f7f-dd7fba1cbf43 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:42.874393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:42.897292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1062/2126] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:43.366520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:43.369218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:43.672550 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199fc19-9984-432b-9d50-dcffd78b2638 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:43.695504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:43.697557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:43.697996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:43.698434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] PATH_INFO: `/v3/projects/c5c53db0f2074b2b90513cce89fe9307` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:43.700125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c53db0f2074b2b90513cce89fe9307)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:43.709584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:43.723369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:43.884079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c53db0f2074b2b90513cce89fe9307', 'request_id': 'req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:43.909071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c53db0f2074b2b90513cce89fe9307', 'request_id': 'req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:43.930636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c53db0f2074b2b90513cce89fe9307', 'request_id': 'req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:43.952512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] The token cache is being invalidate because project c5c53db0f2074b2b90513cce89fe9307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:43.953584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:43.954567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-616ad9ce-c3bf-49f9-bf7b-7b8f34527a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:43.956763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1065/2127] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:43 2022] DELETE /identity/v3/projects/c5c53db0f2074b2b90513cce89fe9307 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:44.703857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:44.705082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:44.705364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:44.754763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] MFA Rules not processed for user `902e421fe49444318dace820aaa881f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:44.756387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:44.830903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0ad9-8084-4092-9801-8860e650e40c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:44.831390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0ad9-8084-4092-9801-8860e650e40c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:44.831492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e0ad9-8084-4092-9801-8860e650e40c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:44.836716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1066/2128] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:30:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:30:44.847437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:44.848732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:44.849147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:44.886618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:44.888134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:44.938327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1063/2129] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:44.950305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:44.951671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:45.086112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1067/2130] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:30:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:45.090823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c91d08-136b-4bae-87b0-57bf66d80598 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:45.099170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:45.101285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:45.112580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:45.115042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:45.115042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:45.115042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] PATH_INFO: `/v3/users/5f4d9c2505f744b4b62fffac21648f0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:45.120778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4d9c2505f744b4b62fffac21648f0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:45.122741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:45.132346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7bf0d-3987-4efb-a54c-d89552960970 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:45.160002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:45.160732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:45.161332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:45.161641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:45.178025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:45.181810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:45.197141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1068/2131] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:45.249650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4d9c2505f744b4b62fffac21648f0b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:45.264056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4d9c2505f744b4b62fffac21648f0b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:45.265162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4d9c2505f744b4b62fffac21648f0b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:45.286644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4d9c2505f744b4b62fffac21648f0b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:45.334781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0c1643-ea9c-4cd7-81be-f55e58cace08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4d9c2505f744b4b62fffac21648f0b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:45.353694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1064/2132] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:44 2022] DELETE /identity/v3/users/5f4d9c2505f744b4b62fffac21648f0b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:45.808470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:45.840712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9627b4-4056-4942-a052-3c9f876fee91 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:46.076901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:46.079031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:46.079688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:46.080835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] PATH_INFO: `/v3/projects/584cd5d9848d48dea5ac9a01dbaaabf1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:46.082508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584cd5d9848d48dea5ac9a01dbaaabf1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:46.085450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:46.133451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:46.232445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584cd5d9848d48dea5ac9a01dbaaabf1', 'request_id': 'req-d7b17304-d132-4509-931a-93a97603f623'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:46.241184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584cd5d9848d48dea5ac9a01dbaaabf1', 'request_id': 'req-d7b17304-d132-4509-931a-93a97603f623'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:46.254789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584cd5d9848d48dea5ac9a01dbaaabf1', 'request_id': 'req-d7b17304-d132-4509-931a-93a97603f623'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:46.280153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] The token cache is being invalidate because project 584cd5d9848d48dea5ac9a01dbaaabf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:46.280647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:46.281383 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7b17304-d132-4509-931a-93a97603f623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:46.283343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1069/2133] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:45 2022] DELETE /identity/v3/projects/584cd5d9848d48dea5ac9a01dbaaabf1 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:46.294785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:46.295696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:46.296280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:46.349968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:46.353643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:46.400894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:46.503396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:46.503617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:46.503867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:46.507190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1065/2134] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:46.537013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:46.538355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:46.556624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1070/2135] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:46.561114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:46.582451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79cece-785c-49bd-9f86-378bb8da575c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:46.589681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:46.590617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:46.591116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:46.591699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:46.592580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:46.593397 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:46.598951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:46.645528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1066/2136] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:46.650936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:46.664061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cefe98-962f-4fbf-b435-5d6373d95907 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:46.672771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:46.673499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:46.673779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:46.674054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:46.674768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:46.678145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:46.775229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1071/2137] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:46 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:46.779200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:46.798382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611643ae-e437-49fa-af19-feb8a921af2c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:46.922315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:46.922956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:46.923374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:46.923687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:46.924363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:46.926315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:46.973460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1067/2138] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:46 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:46.975896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:46.992435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69596699-6aa3-4d86-92d8-7bd489bed996 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.002745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.003726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.004193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.004648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.005930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.008341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.027825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1072/2139] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:47.030171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.050124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02567cc3-7e52-4984-911d-2407c9bf09e4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.061075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417828d1-48d9-4b7d-aece-e69406348fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.061714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417828d1-48d9-4b7d-aece-e69406348fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.062002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417828d1-48d9-4b7d-aece-e69406348fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.062849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417828d1-48d9-4b7d-aece-e69406348fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.063721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417828d1-48d9-4b7d-aece-e69406348fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.065374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417828d1-48d9-4b7d-aece-e69406348fdf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.081395 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1068/2140] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:47.085142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.097146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9233760-136a-4cb6-8d43-5e82ef729696 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.104964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.105758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.106125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.106474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] PATH_INFO: `/v3/projects/79bee8207c814bb6b77999e8cc227955/users/da9185b01b0a4813b72f68d3e7b22d7b/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.134173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bee8207c814bb6b77999e8cc227955, user_id=da9185b01b0a4813b72f68d3e7b22d7b, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.137558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.160373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1073/2141] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:47 2022] PUT /identity/v3/projects/79bee8207c814bb6b77999e8cc227955/users/da9185b01b0a4813b72f68d3e7b22d7b/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:47.184166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.186079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.186371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.291087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] MFA Rules not processed for user `da9185b01b0a4813b72f68d3e7b22d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:47.292480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:47.484960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1069/2142] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:47.486155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.596025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110859a6-532d-4156-b4c1-ad7a3816ca9c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.601179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.601928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.602527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.602653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.603202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.605615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.676678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1074/2143] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:47 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:47.680392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.697748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a685a17-cdcb-436e-b78c-7a2485264a75 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.746865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:47.820540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.821863 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.822153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.822456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.823079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.825063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.857955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1070/2144] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:47 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:30:47.862516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.882498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a95f9a-93c4-475b-94a9-235095b13c2b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.887373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.888026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.888462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.888781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.889964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.892836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.907649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1075/2145] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:47.909728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.920544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176df54d-d524-46ba-a879-c5d88b9e142c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.926070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ef370-888f-4572-95b5-c7c31e07cc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.926755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ef370-888f-4572-95b5-c7c31e07cc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.927079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ef370-888f-4572-95b5-c7c31e07cc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.927443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ef370-888f-4572-95b5-c7c31e07cc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.928282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ef370-888f-4572-95b5-c7c31e07cc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.929947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ef370-888f-4572-95b5-c7c31e07cc53 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.945205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1071/2146] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:30:47.951999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:47.962943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef3ade-9b9c-4e5d-85d2-1d98867ba652 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:47.968078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:47.969300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:47.969578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:47.969844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] PATH_INFO: `/v3/projects/80a9ab83455f4293b643a765e963a2a5/users/14e08f6909164b40a08f234b24787ae2/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:47.986890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a9ab83455f4293b643a765e963a2a5, user_id=14e08f6909164b40a08f234b24787ae2, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:47.989633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:47.991924 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:48.003912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1076/2147] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:47 2022] PUT /identity/v3/projects/80a9ab83455f4293b643a765e963a2a5/users/14e08f6909164b40a08f234b24787ae2/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:48.018101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.018517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.018970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.053249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] MFA Rules not processed for user `14e08f6909164b40a08f234b24787ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:48.054506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:48.152568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1072/2148] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:30:48.160704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.283069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ca2a8-1fb2-4eb8-9cd8-b2bedbbe509e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.288522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.289143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.289628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.293310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.293310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:48.293310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:48.335668 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1077/2149] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:48 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:48.341087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.358525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fdaf2-0a20-458f-b2ba-bbf7db5e82cd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.427710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.428327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.428611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.428884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.429564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:48.431516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:48.461858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1073/2150] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:48 2022] POST /identity/v3/users => generated 296 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:48.465883 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.477079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7449a-de2e-4df6-9a04-50fe5381bd8a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.482063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.482686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.482981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.483278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.484508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:48.486353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:48.498778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1078/2151] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:48.503848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.517644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574425b-d846-4897-8c28-08bdf7bb5dba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.539890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.540836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.541115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.541393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] PATH_INFO: `/v3/projects/caaaa96c9e03418ab806d0eaeeda3ad5/users/110d4fff229a419ca318f85b9c4d8963/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.574487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caaaa96c9e03418ab806d0eaeeda3ad5, user_id=110d4fff229a419ca318f85b9c4d8963, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:48.578138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:48.588201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:48.604976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1074/2152] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:48 2022] PUT /identity/v3/projects/caaaa96c9e03418ab806d0eaeeda3ad5/users/110d4fff229a419ca318f85b9c4d8963/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:48.610779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.625025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0bd6f-5c2b-4cdd-8f0e-c4f3cc42e883 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.697884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74665ebf-8f40-4686-be4c-77f1aa7b4e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.698496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74665ebf-8f40-4686-be4c-77f1aa7b4e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.698775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74665ebf-8f40-4686-be4c-77f1aa7b4e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.699045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74665ebf-8f40-4686-be4c-77f1aa7b4e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.699749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74665ebf-8f40-4686-be4c-77f1aa7b4e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:48.701346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74665ebf-8f40-4686-be4c-77f1aa7b4e2c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:48.713378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1079/2153] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:48.718229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.731868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6344e-9028-4f5b-acbc-5c199dfbdc61 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.738315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.738672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.738961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.739237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] PATH_INFO: `/v3/projects/caaaa96c9e03418ab806d0eaeeda3ad5/users/110d4fff229a419ca318f85b9c4d8963/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.741671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caaaa96c9e03418ab806d0eaeeda3ad5, user_id=110d4fff229a419ca318f85b9c4d8963, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:48.744367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:48.758350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1075/2154] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:48 2022] PUT /identity/v3/projects/caaaa96c9e03418ab806d0eaeeda3ad5/users/110d4fff229a419ca318f85b9c4d8963/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:48.769713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.769943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.770228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.812227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] MFA Rules not processed for user `110d4fff229a419ca318f85b9c4d8963`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:48.813592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:48.874244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:48.902234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1080/2155] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:48.914638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:48.916451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:48.982655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4b977-9e38-4454-a4cc-a73d033fdf10 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:48.986723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:48.987395 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:48.987772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:48.988109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:48.998534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:49.000189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:49.016203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1076/2156] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:49.064368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:49.065877 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:49.079286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ff842-992e-4b53-a9e9-871f93392466 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:49.083295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae021c8-1b4d-4a8a-a4bf-978bbf99dea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:49.084335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae021c8-1b4d-4a8a-a4bf-978bbf99dea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:49.084851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae021c8-1b4d-4a8a-a4bf-978bbf99dea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:49.085259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae021c8-1b4d-4a8a-a4bf-978bbf99dea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:49.165871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae021c8-1b4d-4a8a-a4bf-978bbf99dea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:49.167378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae021c8-1b4d-4a8a-a4bf-978bbf99dea5 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:49.180815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1081/2157] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:52.308043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:52.375248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965be21-63f4-45ac-b54a-9a01f5cbe063 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:52.385145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:52.385810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:52.386090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:52.386378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] PATH_INFO: `/v3/users/da9185b01b0a4813b72f68d3e7b22d7b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:52.387774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9185b01b0a4813b72f68d3e7b22d7b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:52.393824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:52.445591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9185b01b0a4813b72f68d3e7b22d7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:52.453592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9185b01b0a4813b72f68d3e7b22d7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:52.454215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9185b01b0a4813b72f68d3e7b22d7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:52.461872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9185b01b0a4813b72f68d3e7b22d7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:52.475479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9185b01b0a4813b72f68d3e7b22d7b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:52.482047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1077/2158] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:52 2022] DELETE /identity/v3/users/da9185b01b0a4813b72f68d3e7b22d7b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:52.487414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:52.487952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:52.488046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:52.506744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:52.508062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:52.569786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1082/2159] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:52.580055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:52.581587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:52.587830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:52.644784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47fe86-4815-404c-a71e-b4f0f83c1967 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:52.648364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:52.648965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:52.649240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:52.649508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:52.658766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:52.660232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:52.670358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1078/2160] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:52.947597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:52.949258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:52.962464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf534-ff07-485a-83d5-c14a600327bb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:52.967614 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:52.968228 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:52.969216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:52.969716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] PATH_INFO: `/v3/projects/79bee8207c814bb6b77999e8cc227955` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:52.970956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bee8207c814bb6b77999e8cc227955)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:52.973100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:53.025513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bee8207c814bb6b77999e8cc227955', 'request_id': 'req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.031727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bee8207c814bb6b77999e8cc227955', 'request_id': 'req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.039164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bee8207c814bb6b77999e8cc227955', 'request_id': 'req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.051141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] The token cache is being invalidate because project 79bee8207c814bb6b77999e8cc227955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:53.051436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.052149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.053701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1083/2161] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:52 2022] DELETE /identity/v3/projects/79bee8207c814bb6b77999e8cc227955 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:53.058427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:53.068907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528abfd4-8b88-4ee7-85b5-be6c76728d51 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:53.125366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:53.125998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:53.126278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:53.126551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] PATH_INFO: `/v3/users/14e08f6909164b40a08f234b24787ae2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:53.127702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e08f6909164b40a08f234b24787ae2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:53.129697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:53.186025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e08f6909164b40a08f234b24787ae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.193926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e08f6909164b40a08f234b24787ae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.194591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e08f6909164b40a08f234b24787ae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.202013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e08f6909164b40a08f234b24787ae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.216560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e08f6909164b40a08f234b24787ae2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.225875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1079/2162] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:53 2022] DELETE /identity/v3/users/14e08f6909164b40a08f234b24787ae2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:53.588529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:53.590146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:53.610963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2105f3-72ab-4c1f-a6b1-b4e1affb4f4d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:53.619602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:53.685533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:53.686158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:53.686431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:53.686700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] PATH_INFO: `/v3/projects/80a9ab83455f4293b643a765e963a2a5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:53.688500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a9ab83455f4293b643a765e963a2a5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:53.690483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:53.741077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a9ab83455f4293b643a765e963a2a5', 'request_id': 'req-7582e7a9-f761-4564-8539-5a30bbb7c92b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.747873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a9ab83455f4293b643a765e963a2a5', 'request_id': 'req-7582e7a9-f761-4564-8539-5a30bbb7c92b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.757417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a9ab83455f4293b643a765e963a2a5', 'request_id': 'req-7582e7a9-f761-4564-8539-5a30bbb7c92b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.773214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] The token cache is being invalidate because project 80a9ab83455f4293b643a765e963a2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:53.773567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.774260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.776079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1084/2163] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:53 2022] DELETE /identity/v3/projects/80a9ab83455f4293b643a765e963a2a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:53.780222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:53.782473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:53.795311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bea7d-0ec8-4a1f-9311-5b943fd10028 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:53.817420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:53.856935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:53.857553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:53.857827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:53.858094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] PATH_INFO: `/v3/users/110d4fff229a419ca318f85b9c4d8963` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:53.860171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] RBAC: Authorizing `identity:delete_user(user_id=110d4fff229a419ca318f85b9c4d8963)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:53.862094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:53.909512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110d4fff229a419ca318f85b9c4d8963'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.916956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110d4fff229a419ca318f85b9c4d8963'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.917643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110d4fff229a419ca318f85b9c4d8963'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.925567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110d4fff229a419ca318f85b9c4d8963'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.937692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bce370af-02e4-4ea6-bdff-b40fb73da41a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110d4fff229a419ca318f85b9c4d8963'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:53.943804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1080/2164] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:30:53 2022] DELETE /identity/v3/users/110d4fff229a419ca318f85b9c4d8963 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:54.234623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.236245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:54.253798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582e7a9-f761-4564-8539-5a30bbb7c92b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.330525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.331636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.332056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.332376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] PATH_INFO: `/v3/projects/caaaa96c9e03418ab806d0eaeeda3ad5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.334209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] RBAC: Authorizing `identity:delete_project(project_id=caaaa96c9e03418ab806d0eaeeda3ad5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.337269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.393474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:54.397785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caaaa96c9e03418ab806d0eaeeda3ad5', 'request_id': 'req-560f5373-41d9-4b15-80df-8ef16f1d957b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:54.404917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caaaa96c9e03418ab806d0eaeeda3ad5', 'request_id': 'req-560f5373-41d9-4b15-80df-8ef16f1d957b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:54.413147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caaaa96c9e03418ab806d0eaeeda3ad5', 'request_id': 'req-560f5373-41d9-4b15-80df-8ef16f1d957b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:54.425040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] The token cache is being invalidate because project caaaa96c9e03418ab806d0eaeeda3ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:30:54.425265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:54.425888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-560f5373-41d9-4b15-80df-8ef16f1d957b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:30:54.427370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1085/2165] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:30:54 2022] DELETE /identity/v3/projects/caaaa96c9e03418ab806d0eaeeda3ad5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:54.433504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.433791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.434090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.456884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:54.458679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:54.532573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1081/2166] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:54.540889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.541171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.541446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.564998 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:54.566306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:54.578663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1086/2167] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:54.582491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.592710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a77c1f-26ac-49c3-9662-66bbabefcf60 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.597947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.598556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.598842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.599362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.600249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:30:54.600550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.602160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.614603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1082/2168] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:30:54.618983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.628987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eb3e0-367f-4b52-a394-19803b673916 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.633576 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.634178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.634456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.634724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.635354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.637292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.679417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1087/2169] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:54 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:54.680561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.691389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a527-5a8e-4d90-80c8-a426345b8559 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.721171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:54.754846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.755472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.755781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.756053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.756685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.758590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.783413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1083/2170] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:54 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:54.788432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.799731 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00548fe9-c033-4a80-8050-5545ac6b2bd8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.804695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.805467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.805827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.806102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.806745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.808655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.823577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1088/2171] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:54.824835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.835510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47404038-d00a-4f3f-8b72-d24cd8dc046e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.840366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd020df1-7c11-4012-87e3-d1bef2492315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.841037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd020df1-7c11-4012-87e3-d1bef2492315 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.841408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd020df1-7c11-4012-87e3-d1bef2492315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.841771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd020df1-7c11-4012-87e3-d1bef2492315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.842527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd020df1-7c11-4012-87e3-d1bef2492315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.844722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd020df1-7c11-4012-87e3-d1bef2492315 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.855219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1084/2172] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:54.860365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:54.870353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5ffb-900b-4b11-82a8-7397d4c2de76 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:54.875663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:54.876321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.876598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.876869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] PATH_INFO: `/v3/projects/362d83df69704c7fabdf185febbd0c18/users/0ca5fdbb8ec144dd80dab84afb0130b6/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.894908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=362d83df69704c7fabdf185febbd0c18, user_id=0ca5fdbb8ec144dd80dab84afb0130b6, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:54.897863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:54.911726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1089/2173] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:54 2022] PUT /identity/v3/projects/362d83df69704c7fabdf185febbd0c18/users/0ca5fdbb8ec144dd80dab84afb0130b6/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:54.924726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:54.925016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:54.925288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:54.959142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] MFA Rules not processed for user `0ca5fdbb8ec144dd80dab84afb0130b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:54.960444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:55.041944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1085/2174] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:55.046913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.126378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fac1f-3da6-4e2b-8a13-a6c33e62a571 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.131373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.132027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.132307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.132583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.133371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.135290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.165979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1090/2175] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:30:55 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:55.170200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.187938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10426e-f79d-46af-b781-9bf900f69e57 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.258703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.259419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.259790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.260895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.260895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.262995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.286305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1086/2176] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:30:55 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:30:55.290149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.301382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7ce52-8fcf-4047-a239-31efc9bd5952 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.306418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.307019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.307297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.307595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.308243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.309840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.329505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1091/2177] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:55.335664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.349823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47693516-f343-4274-bd47-88d7df957885 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.355927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.356595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.357518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.357807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] PATH_INFO: `/v3/projects/044667717b30485e9927f9b5f8709d21/users/d4dc95869f374913b57f9a531b63b61f/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.388536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=044667717b30485e9927f9b5f8709d21, user_id=d4dc95869f374913b57f9a531b63b61f, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.392427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.411591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1087/2178] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:55 2022] PUT /identity/v3/projects/044667717b30485e9927f9b5f8709d21/users/d4dc95869f374913b57f9a531b63b61f/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:30:55.414365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.425827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99872b5-6561-47b7-bf57-6a355e636bc6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.469792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:55.490932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8698fb-f04c-460d-a513-48e91b9c22b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.491800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8698fb-f04c-460d-a513-48e91b9c22b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.492146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8698fb-f04c-460d-a513-48e91b9c22b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.492476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8698fb-f04c-460d-a513-48e91b9c22b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.493120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8698fb-f04c-460d-a513-48e91b9c22b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.494698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8698fb-f04c-460d-a513-48e91b9c22b9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.504712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1092/2179] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:30:55.508825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.515822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:30:55.519802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdebc32-6945-424e-825a-585a1edf3c0f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.525343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.526014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.526290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.526561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] PATH_INFO: `/v3/projects/044667717b30485e9927f9b5f8709d21/users/d4dc95869f374913b57f9a531b63b61f/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.528506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044667717b30485e9927f9b5f8709d21, user_id=d4dc95869f374913b57f9a531b63b61f, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.532200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.542787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1088/2180] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:30:55 2022] PUT /identity/v3/projects/044667717b30485e9927f9b5f8709d21/users/d4dc95869f374913b57f9a531b63b61f/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:30:55.554346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.554346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.554633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.583459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] MFA Rules not processed for user `d4dc95869f374913b57f9a531b63b61f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:30:55.584715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:55.661549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1093/2181] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:30:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:30:55.673746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.675845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:30:55.738080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f5ff-3206-4af6-957c-5b83141a4ab1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.741742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.742358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.742641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.742914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.752785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.754287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.764594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1089/2182] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:30:55.855916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:30:55.865829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699368b-bb6c-4d8a-8ffa-04aa59bab9e4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:30:55.869376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bdfdf-2d4a-4375-9227-dda76ca0dfac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:30:55.870008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bdfdf-2d4a-4375-9227-dda76ca0dfac service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:30:55.870300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bdfdf-2d4a-4375-9227-dda76ca0dfac service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:30:55.870527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bdfdf-2d4a-4375-9227-dda76ca0dfac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:30:55.927589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bdfdf-2d4a-4375-9227-dda76ca0dfac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:30:55.929031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bdfdf-2d4a-4375-9227-dda76ca0dfac service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:30:55.938020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1094/2183] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:30:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:00.871403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:00.936774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9222f0-ec49-4c3c-9c26-c166d68b95b1 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:00.941505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:00.942135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:00.942415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:00.942686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] PATH_INFO: `/v3/users/0ca5fdbb8ec144dd80dab84afb0130b6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:00.943887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca5fdbb8ec144dd80dab84afb0130b6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:00.945799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:01.001961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca5fdbb8ec144dd80dab84afb0130b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.010935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca5fdbb8ec144dd80dab84afb0130b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.011718 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca5fdbb8ec144dd80dab84afb0130b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.019661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca5fdbb8ec144dd80dab84afb0130b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.031588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca5fdbb8ec144dd80dab84afb0130b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.038047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1090/2184] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:31:00 2022] DELETE /identity/v3/users/0ca5fdbb8ec144dd80dab84afb0130b6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:01.042499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:01.042922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:01.043060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:01.063089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:01.064321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:01.123202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1095/2185] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:01.133649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:01.135043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:01.149328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:01.195511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83729a-f1d2-467c-8e84-500b11ce0c3c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:01.199045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:01.199679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:01.199959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:01.200227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:01.209251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:01.210658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:01.220352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1091/2186] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:01.503276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:01.504882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:01.512545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:01.519104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5abfc4-a341-4ce7-a86f-37d2de10e14e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:01.524433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:01.525174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:01.525458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:01.525734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] PATH_INFO: `/v3/projects/362d83df69704c7fabdf185febbd0c18` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:01.526866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=362d83df69704c7fabdf185febbd0c18)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:01.528918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:01.590165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362d83df69704c7fabdf185febbd0c18', 'request_id': 'req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.597091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362d83df69704c7fabdf185febbd0c18', 'request_id': 'req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.605717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362d83df69704c7fabdf185febbd0c18', 'request_id': 'req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.618573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] The token cache is being invalidate because project 362d83df69704c7fabdf185febbd0c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:31:01.618949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.619702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.621264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1096/2187] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:31:01 2022] DELETE /identity/v3/projects/362d83df69704c7fabdf185febbd0c18 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:01.625399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:01.634998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a11ac1-c151-4b7a-ad91-a27f5c14fa8d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:01.693590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:01.694249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:01.694526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:01.694797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] PATH_INFO: `/v3/users/d4dc95869f374913b57f9a531b63b61f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:01.695953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4dc95869f374913b57f9a531b63b61f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:01.697922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:01.747626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4dc95869f374913b57f9a531b63b61f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.756475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4dc95869f374913b57f9a531b63b61f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.757277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4dc95869f374913b57f9a531b63b61f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.767097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4dc95869f374913b57f9a531b63b61f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.781246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5772e9b7-e699-4d51-8f74-309d47e6af71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4dc95869f374913b57f9a531b63b61f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:01.788019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1092/2188] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:31:01 2022] DELETE /identity/v3/users/d4dc95869f374913b57f9a531b63b61f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:02.245922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.247691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:02.262449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b74bc-dc65-4c8f-87d1-021e7055dde4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.336197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.336860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.337134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.337403 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] PATH_INFO: `/v3/projects/044667717b30485e9927f9b5f8709d21` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.338548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=044667717b30485e9927f9b5f8709d21)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:02.340638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:02.398166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044667717b30485e9927f9b5f8709d21', 'request_id': 'req-78979e8b-8065-459a-a4e2-495fccc18d8e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:02.410361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044667717b30485e9927f9b5f8709d21', 'request_id': 'req-78979e8b-8065-459a-a4e2-495fccc18d8e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:02.425208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044667717b30485e9927f9b5f8709d21', 'request_id': 'req-78979e8b-8065-459a-a4e2-495fccc18d8e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:02.441642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] The token cache is being invalidate because project 044667717b30485e9927f9b5f8709d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:31:02.441642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:02.442332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-78979e8b-8065-459a-a4e2-495fccc18d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:02.444409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1097/2189] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:31:02 2022] DELETE /identity/v3/projects/044667717b30485e9927f9b5f8709d21 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:02.449971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.450338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.450338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.491249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:02.492650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:02.535647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:02.598529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1093/2190] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:02.609110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.609451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.609724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.633709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:02.635097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:02.644895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1098/2191] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:02.649676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.662486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52501774-636a-4dfa-813a-ee17e34a69f2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.668985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.669659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.670053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.670237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.671414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:31:02.671765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:02.673511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:02.688004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1094/2192] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:31:02.692603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.707144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499d39c-b2e3-4021-bbae-e14cb9a8a34d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.713411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.714037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.714322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.714599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.715285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:02.717691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:02.735409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:02.785941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1099/2193] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:31:02 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:02.790356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.801897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc2cd0-c26d-4e25-92e3-f9cd2ca2165a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.864479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.865099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.865384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.865659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.866286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:02.868232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:02.896013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1095/2194] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:31:02 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:02.900176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.911221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251f39b-f437-463f-96a2-698d61522336 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.915790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.916476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.916827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.917191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.917841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:02.919444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:02.930983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1100/2195] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:31:02.934621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.949026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef898551-673e-4b9e-b43d-b52c549267ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.954463 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adf24f-f1b5-4f8e-a872-230249ee085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.955074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adf24f-f1b5-4f8e-a872-230249ee085d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.955354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adf24f-f1b5-4f8e-a872-230249ee085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.955673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adf24f-f1b5-4f8e-a872-230249ee085d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:02.956333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adf24f-f1b5-4f8e-a872-230249ee085d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:02.958126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adf24f-f1b5-4f8e-a872-230249ee085d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:02.971577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1096/2196] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:02.974425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:02.986033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652e9e4-4dda-48ca-a5b0-0208ac709e30 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:02.991226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:02.992409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:02.992409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:02.992637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] PATH_INFO: `/v3/projects/7323a6e535ae48549fce1e8f2e569ace/users/b735eaf967a44445bfaf5555a4ba09e3/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.008311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7323a6e535ae48549fce1e8f2e569ace, user_id=b735eaf967a44445bfaf5555a4ba09e3, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:03.010934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:03.024302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1101/2197] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:31:02 2022] PUT /identity/v3/projects/7323a6e535ae48549fce1e8f2e569ace/users/b735eaf967a44445bfaf5555a4ba09e3/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:03.035197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:03.035467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:03.035796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.065506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] MFA Rules not processed for user `b735eaf967a44445bfaf5555a4ba09e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:03.066811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:03.132683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1097/2198] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:03.144121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:03.145788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:03.204795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6d53-549e-4838-ae7a-d2721d22f5d6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:03.208365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4276c5-0140-41d6-9fd2-d8d801c2369c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:03.208973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4276c5-0140-41d6-9fd2-d8d801c2369c service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:03.209253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4276c5-0140-41d6-9fd2-d8d801c2369c service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:03.209525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4276c5-0140-41d6-9fd2-d8d801c2369c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.217716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4276c5-0140-41d6-9fd2-d8d801c2369c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:03.219699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4276c5-0140-41d6-9fd2-d8d801c2369c service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:03.228618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1102/2199] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:03.247227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:03.280802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:03.310553 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e8c3e-b8cb-42a6-ad0d-71fbdc05851e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:03.315457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:03.316121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:03.316401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:03.316672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] PATH_INFO: `/v3/users/b735eaf967a44445bfaf5555a4ba09e3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.317810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b735eaf967a44445bfaf5555a4ba09e3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:03.319741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:03.370030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b735eaf967a44445bfaf5555a4ba09e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:03.379497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b735eaf967a44445bfaf5555a4ba09e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:03.380183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b735eaf967a44445bfaf5555a4ba09e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:03.387450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b735eaf967a44445bfaf5555a4ba09e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:03.401081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b735eaf967a44445bfaf5555a4ba09e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:03.408017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1098/2200] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:31:03 2022] DELETE /identity/v3/users/b735eaf967a44445bfaf5555a4ba09e3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:03.412543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:03.413064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:03.413064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.448862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:03.450109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:03.514338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1103/2201] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:03.524948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:03.526589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:03.608308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca1ae6-0107-43fd-be1b-2dfb99786121 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:03.612339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efa74e-743c-4de1-af8f-774ed314851b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:03.612948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efa74e-743c-4de1-af8f-774ed314851b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:03.613223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efa74e-743c-4de1-af8f-774ed314851b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:03.613489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efa74e-743c-4de1-af8f-774ed314851b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.622511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efa74e-743c-4de1-af8f-774ed314851b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:03.624522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efa74e-743c-4de1-af8f-774ed314851b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:03.634081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1099/2202] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:03.920559 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:03.922629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:03.938478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6e134-336e-406d-af78-2c4f713ba138 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:03.943892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:03.944812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:03.945180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:03.945624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] PATH_INFO: `/v3/projects/7323a6e535ae48549fce1e8f2e569ace` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:03.946816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7323a6e535ae48549fce1e8f2e569ace)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:03.948789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.003669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7323a6e535ae48549fce1e8f2e569ace', 'request_id': 'req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:04.006261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:04.009121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4058e-26a3-4a3a-8f85-3510cfb45696 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.009421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4058e-26a3-4a3a-8f85-3510cfb45696 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.009729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4058e-26a3-4a3a-8f85-3510cfb45696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.017599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7323a6e535ae48549fce1e8f2e569ace', 'request_id': 'req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:04.033576 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7323a6e535ae48549fce1e8f2e569ace', 'request_id': 'req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:04.039662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-98a4058e-26a3-4a3a-8f85-3510cfb45696 None None] MFA Rules not processed for user `472b6623f57a41568fe7ffce8e40e350`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:04.041202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98a4058e-26a3-4a3a-8f85-3510cfb45696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.055143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] The token cache is being invalidate because project 7323a6e535ae48549fce1e8f2e569ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:31:04.056579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:04.058562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-38d5ec18-591b-4511-abcd-7d3ed4f26a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:04.063664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1104/2203] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:31:03 2022] DELETE /identity/v3/projects/7323a6e535ae48549fce1e8f2e569ace => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:31:04.073203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.073987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.074355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.100153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:04.101380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.103594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:04.116833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1100/2204] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:04.172819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1105/2205] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:04.180518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.180839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.181062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.215902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:04.217150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.227391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1101/2206] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:04.231633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.242227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c08c0-c388-4f15-b9c5-479b2de216ae None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.246937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.247590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.247874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.248149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.248799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:31:04.249181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.250806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.264970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1106/2207] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:31:04.269272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.278719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae634dfb-3d0f-47ae-aec5-b0fea87493f0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.283980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.284582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.284890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.285160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.285793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.287765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.316136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1102/2208] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:04.320310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.331374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638f457-5b18-4ce9-b0aa-cadea87c1e11 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.394025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.394632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.394966 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.395292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.395954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.397856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.422282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1107/2209] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:04.424217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.434465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf414d9-814a-4f32-87b9-175a10c49bba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.441049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.441796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.442191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.442566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.443326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.445122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.456715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1103/2210] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:04.460632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.471967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023796-b5ec-47eb-ab7a-20001bf6acb0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.479201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b9b9e-f29a-4488-9004-9e4d9a51e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.479201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9b9e-f29a-4488-9004-9e4d9a51e786 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.479201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9b9e-f29a-4488-9004-9e4d9a51e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.479201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b9b9e-f29a-4488-9004-9e4d9a51e786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.479871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9b9e-f29a-4488-9004-9e4d9a51e786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.482735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b9b9e-f29a-4488-9004-9e4d9a51e786 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.497692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1108/2211] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:31:04.502473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.513056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff480b7-ceda-41fb-ae80-11fbb73b6977 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.518173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9d6a5-f663-4ebb-b362-be852ec7cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.519042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9d6a5-f663-4ebb-b362-be852ec7cfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.519445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9d6a5-f663-4ebb-b362-be852ec7cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.519813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9d6a5-f663-4ebb-b362-be852ec7cfe1 admin admin] PATH_INFO: `/v3/projects/c76a4d8e2c10495ea87da63f39acfab3/users/b9c6f06123e046a283dcdb278c499ed4/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.536029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9d6a5-f663-4ebb-b362-be852ec7cfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76a4d8e2c10495ea87da63f39acfab3, user_id=b9c6f06123e046a283dcdb278c499ed4, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.539006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9d6a5-f663-4ebb-b362-be852ec7cfe1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.564042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1104/2212] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:31:04 2022] PUT /identity/v3/projects/c76a4d8e2c10495ea87da63f39acfab3/users/b9c6f06123e046a283dcdb278c499ed4/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:31:04.575846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.576288 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.576572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.614461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] MFA Rules not processed for user `b9c6f06123e046a283dcdb278c499ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:04.616202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.657159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:04.681517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.681790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.682060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.698003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1109/2213] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:04.704723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:04.706052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.710020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.711424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.766967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1105/2214] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:31:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:04.779517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07bb2a-59db-44ce-8557-268932f99486 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.785267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bfc22-e4a3-4d37-a2c5-48767ecc4a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.785775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bfc22-e4a3-4d37-a2c5-48767ecc4a3b service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.785967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bfc22-e4a3-4d37-a2c5-48767ecc4a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.786239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bfc22-e4a3-4d37-a2c5-48767ecc4a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.796189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bfc22-e4a3-4d37-a2c5-48767ecc4a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.797768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bfc22-e4a3-4d37-a2c5-48767ecc4a3b service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.808268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1110/2215] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:31:04.822810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:04.824593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:04.906204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7427-8c1f-4f91-801a-f01d08701a06 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:04.912961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:04.913560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:04.913878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:04.914112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:04.924048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:04.925749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:04.936743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1106/2216] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:31:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:31:05.640723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36843e-0f42-47c1-9ad5-94c1cfcf07f3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:05.641225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36843e-0f42-47c1-9ad5-94c1cfcf07f3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:05.641305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36843e-0f42-47c1-9ad5-94c1cfcf07f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:05.664424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3e36843e-0f42-47c1-9ad5-94c1cfcf07f3 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:05.665658 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3e36843e-0f42-47c1-9ad5-94c1cfcf07f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:05.673895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1111/2217] 10.222.0.25 () {58 vars in 967 bytes} [Fri Sep 9 06:31:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:05.688163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:05.699230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffab00-951d-4678-99c0-5859f9b6d7cd service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:05.705110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:05.705719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:05.706001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:05.706274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:05.714573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:05.716181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:05.725324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1107/2218] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:31:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 06:31:05.967862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6af6-bbc6-4118-9632-024d4000a349 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:05.968150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6af6-bbc6-4118-9632-024d4000a349 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:05.968423 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d6af6-bbc6-4118-9632-024d4000a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:05.987597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e94d6af6-bbc6-4118-9632-024d4000a349 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:05.988796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e94d6af6-bbc6-4118-9632-024d4000a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:05.995904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1112/2219] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:31:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:06.009100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:06.020739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c45d76-415b-46cd-a6e9-b681aa60230d service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:06.025812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:06.026415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:06.026698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:06.027043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:06.035171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:06.036773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:06.045010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1108/2220] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:31:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:31:06.414926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:06.415638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:06.415915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:06.435561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:06.436911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:06.447747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1113/2221] 10.222.0.25 () {58 vars in 967 bytes} [Fri Sep 9 06:31:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:06.461763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:06.468611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:06.475337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72505b7-bfac-414c-9250-b61202bb19b6 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:06.480877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4988901-adec-4a59-9aae-c2bad3c0f3e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:06.481482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988901-adec-4a59-9aae-c2bad3c0f3e5 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:06.481762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988901-adec-4a59-9aae-c2bad3c0f3e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:06.482040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988901-adec-4a59-9aae-c2bad3c0f3e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:06.490996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4988901-adec-4a59-9aae-c2bad3c0f3e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:06.492622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4988901-adec-4a59-9aae-c2bad3c0f3e5 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:06.503436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1109/2222] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:31:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 06:31:10.863535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:10.867571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:10.993795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d47ed-6ef0-457e-b0ad-0b2d3cfc0b44 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:11.020150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:11.021422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.022520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.023008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] PATH_INFO: `/v3/users/902e421fe49444318dace820aaa881f2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.025464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] RBAC: Authorizing `identity:delete_user(user_id=902e421fe49444318dace820aaa881f2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:11.037113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:11.146057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902e421fe49444318dace820aaa881f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.175083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902e421fe49444318dace820aaa881f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.176173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902e421fe49444318dace820aaa881f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.177377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902e421fe49444318dace820aaa881f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.185033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:11.191617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902e421fe49444318dace820aaa881f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.238597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1114/2223] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:31:10 2022] DELETE /identity/v3/users/902e421fe49444318dace820aaa881f2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:11.243168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.243650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.243740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.269998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:11.271450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:11.373345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1110/2224] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:11.380346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:11.382025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:11.462276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42883b89-805c-4bd6-9764-99a52803f25e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:11.467950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77cb90-32a3-488f-8fa5-3d39a62a32a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:11.468684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77cb90-32a3-488f-8fa5-3d39a62a32a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.469068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77cb90-32a3-488f-8fa5-3d39a62a32a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.469496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77cb90-32a3-488f-8fa5-3d39a62a32a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.490663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77cb90-32a3-488f-8fa5-3d39a62a32a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:11.492354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77cb90-32a3-488f-8fa5-3d39a62a32a1 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:11.503041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1115/2225] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:11.726001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:11.728707 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:11.744687 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761847ab-edf0-43b8-88ea-3af15d56d6d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:11.769917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:11.770918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.771586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.772117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] PATH_INFO: `/v3/projects/6ceb527ce5a74fc98bb7ca6fcb0fabba` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.773379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ceb527ce5a74fc98bb7ca6fcb0fabba)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:11.775575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:11.825814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ceb527ce5a74fc98bb7ca6fcb0fabba', 'request_id': 'req-4167508c-db9a-487a-bab0-617ec6abfde3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.834095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ceb527ce5a74fc98bb7ca6fcb0fabba', 'request_id': 'req-4167508c-db9a-487a-bab0-617ec6abfde3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.839920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ceb527ce5a74fc98bb7ca6fcb0fabba', 'request_id': 'req-4167508c-db9a-487a-bab0-617ec6abfde3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.851535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] The token cache is being invalidate because project 6ceb527ce5a74fc98bb7ca6fcb0fabba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:31:11.851851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.852489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4167508c-db9a-487a-bab0-617ec6abfde3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:11.853976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1111/2226] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:31:11 2022] DELETE /identity/v3/projects/6ceb527ce5a74fc98bb7ca6fcb0fabba => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:11.858953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.859235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.859508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.879145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:11.880364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:11.938787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1116/2227] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:11.947173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.947533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.947820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.968084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:11.969338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:11.977359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1112/2228] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:11.981638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:11.991506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6bc18-4d6e-4513-982a-911f513e8481 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:11.996114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:11.996963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:11.997243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:11.997521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:11.998231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:31:11.999054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.000650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.015249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1117/2229] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:31:12.021073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:12.030547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b4f3-3e0b-440f-9328-4f0100a81985 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:12.035598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:12.036201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.036476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.036742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.037385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.039362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.070445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1113/2230] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:31:12 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:12.074434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:12.086263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee682121-10fc-4eba-ab26-d8cea1367844 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:12.155993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:12.156720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.157072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.157438 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.158137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.160256 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.172644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:12.187287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1118/2231] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:31:12 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:31:12.190424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:12.202381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c71c-113b-4ea5-8b43-351c9d81b6d3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:12.207481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:12.208443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.208588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.208750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.209548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.227222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.227222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1114/2232] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:12.228378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:12.239584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61115cf-bf45-4fdf-a960-4d3261ddd348 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:12.244289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176ea4b-e64b-487b-9036-a84346ae5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:12.244894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176ea4b-e64b-487b-9036-a84346ae5e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.245469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176ea4b-e64b-487b-9036-a84346ae5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.245749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176ea4b-e64b-487b-9036-a84346ae5e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.246397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176ea4b-e64b-487b-9036-a84346ae5e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.248114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176ea4b-e64b-487b-9036-a84346ae5e4b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.259814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1119/2233] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:12.264636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:12.278572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb6c55-cbd6-4281-956d-9261d85ebf4b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:12.285656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:12.286329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.286853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.287139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] PATH_INFO: `/v3/projects/9c96a5a4edc84b70a31c6b0cd2a8f864/users/4659e9d9c65e436aa101db62a801a5fe/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.312366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c96a5a4edc84b70a31c6b0cd2a8f864, user_id=4659e9d9c65e436aa101db62a801a5fe, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.312366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.328315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1115/2234] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:31:12 2022] PUT /identity/v3/projects/9c96a5a4edc84b70a31c6b0cd2a8f864/users/4659e9d9c65e436aa101db62a801a5fe/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:31:12.333520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6287e0e-f48c-44ac-974d-ed707d2fa232 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.333841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6287e0e-f48c-44ac-974d-ed707d2fa232 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.334292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6287e0e-f48c-44ac-974d-ed707d2fa232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.356509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c6287e0e-f48c-44ac-974d-ed707d2fa232 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:12.357741 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c6287e0e-f48c-44ac-974d-ed707d2fa232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:12.441857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1120/2235] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:12.451726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:12.453709 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:12.465018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:12.522352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3010-2e50-4ccd-8078-2f2f6dfd52e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:12.526409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:12.527018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:12.527936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:12.528342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:12.539578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:12.541141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:12.558240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1116/2236] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:13.289625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:13.290145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:13.290145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:13.319992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] MFA Rules not processed for user `4659e9d9c65e436aa101db62a801a5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:13.321399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:13.408692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1121/2237] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:13.429022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:13.430656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:13.507822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b96d1-c644-489a-bfac-de861f47398d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:13.511722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a6522-3117-467d-aa57-773a168bab1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:13.512332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a6522-3117-467d-aa57-773a168bab1e service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:13.512610 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a6522-3117-467d-aa57-773a168bab1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:13.512877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a6522-3117-467d-aa57-773a168bab1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:13.530797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a6522-3117-467d-aa57-773a168bab1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:13.533798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a6522-3117-467d-aa57-773a168bab1e service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:13.549120 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1117/2238] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:13.774574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:13.776126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:13.791408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a40107-5d2b-42ec-9873-368b9e7d4837 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:13.797134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:13.797917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:13.798204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:13.798512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] PATH_INFO: `/v3/users/b9c6f06123e046a283dcdb278c499ed4` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:13.799699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c6f06123e046a283dcdb278c499ed4)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:13.801927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:13.913138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c6f06123e046a283dcdb278c499ed4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:13.924957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c6f06123e046a283dcdb278c499ed4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:13.926037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c6f06123e046a283dcdb278c499ed4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:13.926780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c6f06123e046a283dcdb278c499ed4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:13.938937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c6f06123e046a283dcdb278c499ed4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:13.945408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:13.966600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1122/2239] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:31:13 2022] DELETE /identity/v3/users/b9c6f06123e046a283dcdb278c499ed4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:31:13.971314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:13.971623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:13.971892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:13.992868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:13.994093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:14.056866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1118/2240] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:14.067300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.068746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:14.126534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7065c4-2e38-4255-ae40-fb5d3b12c662 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.130247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be7887-ec0b-42d9-8a74-098a45a87ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:14.130856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7887-ec0b-42d9-8a74-098a45a87ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.131134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7887-ec0b-42d9-8a74-098a45a87ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.131408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7887-ec0b-42d9-8a74-098a45a87ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.141876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be7887-ec0b-42d9-8a74-098a45a87ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:14.143314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be7887-ec0b-42d9-8a74-098a45a87ceb service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:14.153391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1123/2241] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:14.435681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.437243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:14.449325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f680f1-c840-4184-9da0-d146ec3fb895 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.454534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:14.455171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.455461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.455750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] PATH_INFO: `/v3/projects/c76a4d8e2c10495ea87da63f39acfab3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.456877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76a4d8e2c10495ea87da63f39acfab3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:14.458797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:14.506113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76a4d8e2c10495ea87da63f39acfab3', 'request_id': 'req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:14.507769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:14.515036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76a4d8e2c10495ea87da63f39acfab3', 'request_id': 'req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:14.521185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76a4d8e2c10495ea87da63f39acfab3', 'request_id': 'req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:14.535176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] The token cache is being invalidate because project c76a4d8e2c10495ea87da63f39acfab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:31:14.535466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:14.536265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1cdf0cb-7316-410c-adce-5fb03fb1a31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:14.537735 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1119/2242] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:31:14 2022] DELETE /identity/v3/projects/c76a4d8e2c10495ea87da63f39acfab3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:14.543168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.543641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.543722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.566852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:14.568129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:14.632138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1124/2243] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:14.640072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.640365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.640640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.687891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:14.689094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:14.703598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1120/2244] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:14.705134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.716307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b55c2-2bb5-4b78-9eb8-ad5bba999eb7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.723474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:14.724521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.725442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.725717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.726410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:31:14.726730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:14.728632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:14.744577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1125/2245] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:31:14.749011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.761169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ecad5-c5e5-45db-8ebb-be0f10fad1b4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.768100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:14.768757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.769065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.769617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.770104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:14.772398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:14.805661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1121/2246] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:31:14 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:31:14.810282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.821341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436cadb-090b-4e08-9555-298765b3c780 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.883195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:14.883907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.884261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.884540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.885174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:14.887113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:14.912981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1126/2247] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:31:14 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:31:14.921891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.943402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df930e-ba2d-4af1-b331-acad01a7f62a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.949134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04071ff6-4300-425d-b1f4-8fc8befec2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:14.950052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04071ff6-4300-425d-b1f4-8fc8befec2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.950880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04071ff6-4300-425d-b1f4-8fc8befec2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.951623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04071ff6-4300-425d-b1f4-8fc8befec2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.954242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04071ff6-4300-425d-b1f4-8fc8befec2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:14.956584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04071ff6-4300-425d-b1f4-8fc8befec2c3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:14.980096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1122/2248] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:14.981048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:14.990695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:14.991373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:14.991373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:14.992488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538dc830-f256-47d1-9b3e-543329554372 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:14.999337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:15.000100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:15.000381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:15.000651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:15.001294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:15.002904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:15.004370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:15.012092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] MFA Rules not processed for user `4659e9d9c65e436aa101db62a801a5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:15.013303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:15.017891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1127/2249] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:15.021543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:15.032370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be220a-d844-49b7-be24-1e9677a48588 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:15.037143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7435bc8-f900-44bb-8af4-a1e9ccaa098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:15.037812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7435bc8-f900-44bb-8af4-a1e9ccaa098f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:15.038088 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7435bc8-f900-44bb-8af4-a1e9ccaa098f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:15.038359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7435bc8-f900-44bb-8af4-a1e9ccaa098f admin admin] PATH_INFO: `/v3/projects/b7ac9faa5a464413a117cddce7dd8941/users/704270dfa614474ca6afc013a2413bc4/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:15.056869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7435bc8-f900-44bb-8af4-a1e9ccaa098f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ac9faa5a464413a117cddce7dd8941, user_id=704270dfa614474ca6afc013a2413bc4, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:15.059752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7435bc8-f900-44bb-8af4-a1e9ccaa098f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:15.074644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1128/2250] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:31:15 2022] PUT /identity/v3/projects/b7ac9faa5a464413a117cddce7dd8941/users/704270dfa614474ca6afc013a2413bc4/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:31:15.078974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1123/2251] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:15.084667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d10a1-e015-4c56-90fd-f4b983dbad1c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:15.084949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d10a1-e015-4c56-90fd-f4b983dbad1c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:15.085222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d10a1-e015-4c56-90fd-f4b983dbad1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:15.116179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b99d10a1-e015-4c56-90fd-f4b983dbad1c None None] MFA Rules not processed for user `704270dfa614474ca6afc013a2413bc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:15.117490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b99d10a1-e015-4c56-90fd-f4b983dbad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:15.191165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1129/2252] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:15.206122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:15.207772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:15.266690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef581e-2a7b-421f-ba3f-e88063f99abe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:15.318083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c15c1c-06f7-493d-a21a-78504b0fdeaa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:15.318686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c15c1c-06f7-493d-a21a-78504b0fdeaa service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:15.318962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c15c1c-06f7-493d-a21a-78504b0fdeaa service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:15.319233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c15c1c-06f7-493d-a21a-78504b0fdeaa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:15.329039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c15c1c-06f7-493d-a21a-78504b0fdeaa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:15.330183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c15c1c-06f7-493d-a21a-78504b0fdeaa service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:15.339828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1124/2253] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:31:19.196298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:19.196805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:19.196965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:19.217942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] MFA Rules not processed for user `925d5f23bd3f4b35a66736d2a6a3c3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:19.219294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:19.302783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1130/2254] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:23.828606 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8860393-e261-403a-89b4-7508502ec0ee None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:23.829244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8860393-e261-403a-89b4-7508502ec0ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:23.829244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8860393-e261-403a-89b4-7508502ec0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:23.836849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:23.854252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b8860393-e261-403a-89b4-7508502ec0ee None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:23.854780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b8860393-e261-403a-89b4-7508502ec0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:23.921004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1125/2255] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:31:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:23.935275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:23.936824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:24.015125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb513b02-05cf-495b-90ca-26c231df1805 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:24.020313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:24.020928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:24.021233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:24.021460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:24.030318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:24.033579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:24.035537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:24.045478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1131/2256] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:31:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 06:31:29.170959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71816ee8-29da-4731-a35e-99d0219b85e1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:29.171514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71816ee8-29da-4731-a35e-99d0219b85e1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:29.171514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71816ee8-29da-4731-a35e-99d0219b85e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:29.208766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-71816ee8-29da-4731-a35e-99d0219b85e1 None None] MFA Rules not processed for user `3e2bdc73390a43ff9eccc57948e2b4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:29.210408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-71816ee8-29da-4731-a35e-99d0219b85e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:29.327022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1126/2257] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:30.569148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:30.571933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:30.660060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416df90-edce-4c6a-b183-5449e3d721de service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:30.664300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:30.665063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:30.665583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:30.666107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:30.668166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:30.762486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:30.762486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc931ae2-d5be-4c2e-b2d8-373ad0c8aec4 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:30.776145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1132/2258] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:39.319460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=49 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:40.767439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=52 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:47.052868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4b99a-99d4-40aa-ac38-68468c4b2d62 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.053460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4b99a-99d4-40aa-ac38-68468c4b2d62 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.053460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4b99a-99d4-40aa-ac38-68468c4b2d62 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.054874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1127/2259] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:31:47.062200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2a3f2-604d-4919-a430-a64741576042 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.062693 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2a3f2-604d-4919-a430-a64741576042 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.062789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2a3f2-604d-4919-a430-a64741576042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.083431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-18a2a3f2-604d-4919-a430-a64741576042 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:47.084710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18a2a3f2-604d-4919-a430-a64741576042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:47.154780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1133/2260] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:47.166953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4b99a-99d4-40aa-ac38-68468c4b2d62 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:47.168748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-09f4b99a-99d4-40aa-ac38-68468c4b2d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:47.185040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4b99a-99d4-40aa-ac38-68468c4b2d62 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:47.189498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619304b-be2e-4b69-848d-10b48c9bd753 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:47.190294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619304b-be2e-4b69-848d-10b48c9bd753 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.190983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619304b-be2e-4b69-848d-10b48c9bd753 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.191406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619304b-be2e-4b69-848d-10b48c9bd753 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.205797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619304b-be2e-4b69-848d-10b48c9bd753 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:47.207256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619304b-be2e-4b69-848d-10b48c9bd753 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:47.220275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1128/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:47.291455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed5448-c9c6-4a96-b80e-f358ddf25642 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.292040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed5448-c9c6-4a96-b80e-f358ddf25642 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.292396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed5448-c9c6-4a96-b80e-f358ddf25642 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.293977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1134/2262] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:31:47.301008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78abf62-35a7-43f6-ad4c-10b677d76e31 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.301293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78abf62-35a7-43f6-ad4c-10b677d76e31 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.301565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78abf62-35a7-43f6-ad4c-10b677d76e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.321538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d78abf62-35a7-43f6-ad4c-10b677d76e31 None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:47.322737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d78abf62-35a7-43f6-ad4c-10b677d76e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:47.330071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1129/2263] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:47.342512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed5448-c9c6-4a96-b80e-f358ddf25642 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:47.353725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed5448-c9c6-4a96-b80e-f358ddf25642 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:47.357312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693dbd7-3602-4d8c-b047-d97563e5b182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:47.358467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693dbd7-3602-4d8c-b047-d97563e5b182 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.358742 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693dbd7-3602-4d8c-b047-d97563e5b182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.359126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693dbd7-3602-4d8c-b047-d97563e5b182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.366879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693dbd7-3602-4d8c-b047-d97563e5b182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:47.368450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693dbd7-3602-4d8c-b047-d97563e5b182 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:47.378319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1135/2264] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:47.468155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62433026-854e-4524-951f-861b7993f1a2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.468525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62433026-854e-4524-951f-861b7993f1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.468860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62433026-854e-4524-951f-861b7993f1a2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.471105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1130/2265] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:31:47.478941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.479458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.480321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.503159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:47.504784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:47.518156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1136/2266] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:47.529494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62433026-854e-4524-951f-861b7993f1a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:47.548833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62433026-854e-4524-951f-861b7993f1a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:47.548833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0b9f6-ceed-4ea3-a34a-d220028502c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:47.549581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b9f6-ceed-4ea3-a34a-d220028502c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:47.549921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b9f6-ceed-4ea3-a34a-d220028502c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:47.550259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b9f6-ceed-4ea3-a34a-d220028502c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:47.562828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0b9f6-ceed-4ea3-a34a-d220028502c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:47.575270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0b9f6-ceed-4ea3-a34a-d220028502c1 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:47.590344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1131/2267] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:31:50.147328 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:50.219605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f9a83-5094-4207-99a0-e385caf78903 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:50.224486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:50.225121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:50.225398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:50.225757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] PATH_INFO: `/v3/users/704270dfa614474ca6afc013a2413bc4` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:50.226782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] RBAC: Authorizing `identity:delete_user(user_id=704270dfa614474ca6afc013a2413bc4)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:50.228736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:50.281268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704270dfa614474ca6afc013a2413bc4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:50.289714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704270dfa614474ca6afc013a2413bc4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:50.290361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704270dfa614474ca6afc013a2413bc4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:50.290964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704270dfa614474ca6afc013a2413bc4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:50.299085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704270dfa614474ca6afc013a2413bc4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:50.306885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:50.322220 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1137/2268] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:31:50 2022] DELETE /identity/v3/users/704270dfa614474ca6afc013a2413bc4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:50.326885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:50.327377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:50.327499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:50.347355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:50.348908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:50.417655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1132/2269] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:50.425123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:50.426901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:50.501044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e537cd8-e662-4c1d-9fb3-bd9120f0e90a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:50.505300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6d833-bc26-464e-a39b-de9e9eeaca32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:50.505923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d833-bc26-464e-a39b-de9e9eeaca32 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:50.506202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d833-bc26-464e-a39b-de9e9eeaca32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:50.506473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d833-bc26-464e-a39b-de9e9eeaca32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:50.517107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6d833-bc26-464e-a39b-de9e9eeaca32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:50.518669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6d833-bc26-464e-a39b-de9e9eeaca32 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:50.529095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1138/2270] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:31:50.895403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:50.898013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:50.933286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5385470-69fa-438d-8cd0-70673a5930a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:50.947315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:50.950234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:50.950997 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:50.951405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] PATH_INFO: `/v3/projects/b7ac9faa5a464413a117cddce7dd8941` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:50.953064 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ac9faa5a464413a117cddce7dd8941)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:50.956448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.042606 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ac9faa5a464413a117cddce7dd8941', 'request_id': 'req-e0a807a5-5505-4257-a02e-29c6747929e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:51.052210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ac9faa5a464413a117cddce7dd8941', 'request_id': 'req-e0a807a5-5505-4257-a02e-29c6747929e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:51.061272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ac9faa5a464413a117cddce7dd8941', 'request_id': 'req-e0a807a5-5505-4257-a02e-29c6747929e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:51.075838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] The token cache is being invalidate because project b7ac9faa5a464413a117cddce7dd8941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:31:51.077084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:51.078146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0a807a5-5505-4257-a02e-29c6747929e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:31:51.080935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1133/2271] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:31:50 2022] DELETE /identity/v3/projects/b7ac9faa5a464413a117cddce7dd8941 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:51.086637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.086637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.086637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.112350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:51.113581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:51.240026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1139/2272] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:31:51.250914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.251198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.253313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.292106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:51.293420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:51.302117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1134/2273] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:51.306413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.316848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b2ad9-0c76-4b37-8993-68e17def2917 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:51.322493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:51.323258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.323566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.323839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.324689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:31:51.324997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:51.328107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.343574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1140/2274] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:31:51.348263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.361570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d0eb5-52b5-4529-806e-a73cfc370815 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:51.382630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:51.383944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.384229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.384505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.385159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:51.399746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.459481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1135/2275] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:31:51 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:51.462208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.478768 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950edb-3a9a-4104-abfb-46ddbdf97d08 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:51.655951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:51.656635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.656917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.657189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.657949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:51.660319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.695873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1141/2276] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:31:51 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:31:51.697363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.732065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b57cf-25d7-4051-9b08-e4bc1ed31a4f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:51.742799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:51.743668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.744094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.744391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.745087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:51.746793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.760771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1136/2277] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:31:51.765015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.777167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b24d-eac2-4935-bc1f-4df72982629f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:51.782884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d7686-182f-431a-91b6-f31149eb1073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:51.783743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d7686-182f-431a-91b6-f31149eb1073 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.784027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d7686-182f-431a-91b6-f31149eb1073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.784302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d7686-182f-431a-91b6-f31149eb1073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.784961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d7686-182f-431a-91b6-f31149eb1073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:51.787138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d7686-182f-431a-91b6-f31149eb1073 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.793066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:51.802867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1142/2278] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:31:51.804264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.815600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc87d5e-2ea9-4cdf-ad8f-da24481e84cd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:51.820164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:51.820827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.821105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.821385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] PATH_INFO: `/v3/projects/f268b72c7f1a4b69956d2f0805ad3bcf/users/406589819d204b9aaa97040b0b830639/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.830984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:31:51.838444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f268b72c7f1a4b69956d2f0805ad3bcf, user_id=406589819d204b9aaa97040b0b830639, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:51.841732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:51.859579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1137/2279] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:31:51 2022] PUT /identity/v3/projects/f268b72c7f1a4b69956d2f0805ad3bcf/users/406589819d204b9aaa97040b0b830639/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:31:51.869953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514961-7507-4f37-8a33-63dc0b0b6f91 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:51.870236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514961-7507-4f37-8a33-63dc0b0b6f91 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:51.870506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514961-7507-4f37-8a33-63dc0b0b6f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:51.900695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-84514961-7507-4f37-8a33-63dc0b0b6f91 None None] MFA Rules not processed for user `406589819d204b9aaa97040b0b830639`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:31:51.902028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-84514961-7507-4f37-8a33-63dc0b0b6f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:51.968724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1143/2280] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:31:51.982049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:31:51.983865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:31:52.046212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750aa33-8c4d-4a8a-aec3-cf0dc33202d3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:31:52.049890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:31:52.050504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:31:52.050785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:31:52.051059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:31:52.059846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:31:52.061559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:31:52.070370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1138/2281] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:32:01.965105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=32 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:02.066383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:06.201394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:06.201910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:06.202016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:06.221823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:32:06.223197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:06.281461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1144/2282] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:32:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:32:06.303936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:06.305379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:06.375568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c06fcf-2380-4f53-b7d1-e44e6cd44460 service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:06.381121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:06.381730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:06.382012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:06.382285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:06.390282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:06.391908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:06.399919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1139/2283] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:32:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:32:16.274808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=25 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:16.396813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=32 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:24.797174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:24.798781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:24.871443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbc1f2-9b3e-4bb1-b750-678bf3db07a5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:24.876270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:24.877110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:24.877390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:24.877678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:24.879344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:24.890844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:24.893090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b427106-f51e-432f-8292-74301dd8d6ac service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:24.902738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1145/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:32:29.617263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:29.678637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe09edc-b5be-4ca9-b5b2-bcaf9ae22b7c service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:29.683054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:29.683704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:29.683980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:29.684249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] PATH_INFO: `/v3/users/406589819d204b9aaa97040b0b830639` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:29.685347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] RBAC: Authorizing `identity:delete_user(user_id=406589819d204b9aaa97040b0b830639)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:29.687241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:29.736434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406589819d204b9aaa97040b0b830639'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:29.743859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406589819d204b9aaa97040b0b830639'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:29.744501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406589819d204b9aaa97040b0b830639'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:29.751956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406589819d204b9aaa97040b0b830639'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:29.763916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406589819d204b9aaa97040b0b830639'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:29.770400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1140/2285] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:32:29 2022] DELETE /identity/v3/users/406589819d204b9aaa97040b0b830639 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:32:29.774977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd011-c54d-4362-98c1-75ed172706de None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:29.775456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd011-c54d-4362-98c1-75ed172706de None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:29.775578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd011-c54d-4362-98c1-75ed172706de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:29.795008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-73afd011-c54d-4362-98c1-75ed172706de None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:32:29.796218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-73afd011-c54d-4362-98c1-75ed172706de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:29.854215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1146/2286] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:32:29.864048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:29.865422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:29.922483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e46dd-0042-439d-87b7-db9c5fea983c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:29.926345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75906ef6-630b-48d2-879a-709d5db01349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:29.926953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75906ef6-630b-48d2-879a-709d5db01349 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:29.927231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75906ef6-630b-48d2-879a-709d5db01349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:29.927502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75906ef6-630b-48d2-879a-709d5db01349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:29.936897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75906ef6-630b-48d2-879a-709d5db01349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:29.938345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75906ef6-630b-48d2-879a-709d5db01349 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:29.948634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1141/2287] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:32:30.304959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afd011-c54d-4362-98c1-75ed172706de None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.306389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-73afd011-c54d-4362-98c1-75ed172706de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:30.319324 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afd011-c54d-4362-98c1-75ed172706de None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.324107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.324736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.325011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.325280 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] PATH_INFO: `/v3/projects/f268b72c7f1a4b69956d2f0805ad3bcf` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.326394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f268b72c7f1a4b69956d2f0805ad3bcf)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.328338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.378651 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f268b72c7f1a4b69956d2f0805ad3bcf', 'request_id': 'req-1e7b1254-4523-480c-a7c9-84a329b26eb9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:30.384303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f268b72c7f1a4b69956d2f0805ad3bcf', 'request_id': 'req-1e7b1254-4523-480c-a7c9-84a329b26eb9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:30.391521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f268b72c7f1a4b69956d2f0805ad3bcf', 'request_id': 'req-1e7b1254-4523-480c-a7c9-84a329b26eb9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:30.403628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] The token cache is being invalidate because project f268b72c7f1a4b69956d2f0805ad3bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:32:30.403915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:30.404542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e7b1254-4523-480c-a7c9-84a329b26eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:32:30.406011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1147/2288] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:32:30 2022] DELETE /identity/v3/projects/f268b72c7f1a4b69956d2f0805ad3bcf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:32:30.410950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f026f-4183-4813-bb05-2469bc57012f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.411255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f026f-4183-4813-bb05-2469bc57012f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.411525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f026f-4183-4813-bb05-2469bc57012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.430849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-848f026f-4183-4813-bb05-2469bc57012f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:32:30.432234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-848f026f-4183-4813-bb05-2469bc57012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:30.449801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:30.493408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1142/2289] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:32:30.502313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.502596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.502874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.523049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:32:30.524283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:30.531981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1148/2290] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:32:30.536297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f026f-4183-4813-bb05-2469bc57012f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.545960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f026f-4183-4813-bb05-2469bc57012f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.550920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.551530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.551842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.552118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.552827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:32:30.553133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.554723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.567034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1143/2291] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:32:30.571382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.581038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c5ec5-9c33-405f-a0f7-47c77bbd70b2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.585839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.586449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.586759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.587031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.587712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.589629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.594063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:30.618841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1149/2292] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:32:30 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:32:30.623153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.633534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad67bd-2923-4872-b243-7c7bdb75e53e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.690987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.691628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.691909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.692201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.693514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.694794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.715919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1144/2293] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:32:30 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:32:30.720922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.732326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f28d5-4bca-4709-878e-d23cb84ccbda admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.738371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.738971 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.739263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.739519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.740191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.741767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.752469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1150/2294] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:32:30.756509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.766332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a8f51-151f-48cf-b270-49111bc7b748 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.771379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ef4b3-987c-457d-978e-dd103042b0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.772165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ef4b3-987c-457d-978e-dd103042b0fa admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.772445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ef4b3-987c-457d-978e-dd103042b0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.772715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ef4b3-987c-457d-978e-dd103042b0fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.773352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ef4b3-987c-457d-978e-dd103042b0fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.774920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ef4b3-987c-457d-978e-dd103042b0fa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.786404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1145/2295] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:32:30.790302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.800531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5acd6c-5ecd-4d38-94dd-a7b4896d0fa1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:30.804992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:30.805629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.805904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.806176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] PATH_INFO: `/v3/projects/6142fb9ed8a3428ca2216368d073a13c/users/98f34a0b1d4249b2a0c8a27144024974/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.821507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6142fb9ed8a3428ca2216368d073a13c, user_id=98f34a0b1d4249b2a0c8a27144024974, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:30.824164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:30.837246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1151/2296] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:32:30 2022] PUT /identity/v3/projects/6142fb9ed8a3428ca2216368d073a13c/users/98f34a0b1d4249b2a0c8a27144024974/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:32:30.848836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ff96d-61f6-42d6-bb6a-eefae3865c05 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:30.849114 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ff96d-61f6-42d6-bb6a-eefae3865c05 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:30.849379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ff96d-61f6-42d6-bb6a-eefae3865c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:30.876831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-954ff96d-61f6-42d6-bb6a-eefae3865c05 None None] MFA Rules not processed for user `98f34a0b1d4249b2a0c8a27144024974`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:32:30.878047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-954ff96d-61f6-42d6-bb6a-eefae3865c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:30.940849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1146/2297] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:32:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:32:30.952207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:32:30.953896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:32:31.011596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381458-7e32-4ccb-ae24-825a7b0cd775 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:32:31.015766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:32:31.016485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:32:31.017336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:32:31.017763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:32:31.028210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:32:31.029705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:32:31.039325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1152/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:32:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:32:40.937566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:32:41.035172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:33:00.031775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead09a4-c473-4509-86ff-145ae5381819 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:00.032317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead09a4-c473-4509-86ff-145ae5381819 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:00.032434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead09a4-c473-4509-86ff-145ae5381819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:00.053616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9ead09a4-c473-4509-86ff-145ae5381819 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:33:00.055175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9ead09a4-c473-4509-86ff-145ae5381819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:00.133622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1147/2299] 10.222.0.25 () {58 vars in 967 bytes} [Fri Sep 9 06:33:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:33:00.148036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:33:00.149500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:00.230584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2e9f3-8588-4e33-8f4a-c86ae81a80a3 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:33:00.235817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:33:00.236444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:00.236719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:00.236987 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:00.246141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:33:00.247871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:33:00.258161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1153/2300] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:33:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:33:10.125266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:33:10.252933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=32 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:33:12.422169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d276e53-da47-4ac5-bf7f-255b1938b2fd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:12.422659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d276e53-da47-4ac5-bf7f-255b1938b2fd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:12.422754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d276e53-da47-4ac5-bf7f-255b1938b2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:12.444430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3d276e53-da47-4ac5-bf7f-255b1938b2fd None None] MFA Rules not processed for user `925d5f23bd3f4b35a66736d2a6a3c3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:33:12.445694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d276e53-da47-4ac5-bf7f-255b1938b2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:12.502352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1148/2301] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:33:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:33:18.053289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:33:18.119349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c669993-2b58-46ec-af18-f8a963ed92cc service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:33:18.124038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:33:18.124688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:18.124965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:18.125239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] PATH_INFO: `/v3/users/98f34a0b1d4249b2a0c8a27144024974` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:18.126363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f34a0b1d4249b2a0c8a27144024974)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:33:18.128309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:33:18.183529 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f34a0b1d4249b2a0c8a27144024974'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.190841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f34a0b1d4249b2a0c8a27144024974'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.191509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f34a0b1d4249b2a0c8a27144024974'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.192189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f34a0b1d4249b2a0c8a27144024974'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.200586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f34a0b1d4249b2a0c8a27144024974'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.219776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1154/2302] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:33:18 2022] DELETE /identity/v3/users/98f34a0b1d4249b2a0c8a27144024974 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:33:18.225365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:18.225864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:18.225979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:18.245047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:33:18.246259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:18.313014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1149/2303] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:33:18.322880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:33:18.324443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:18.387489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b703a89-5ded-48e1-b9bc-58a28a2543ff admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:33:18.391663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca7582-18d4-4cfa-94fd-6f9af33f2acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:33:18.391967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7582-18d4-4cfa-94fd-6f9af33f2acb service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:18.392250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7582-18d4-4cfa-94fd-6f9af33f2acb service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:18.392519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7582-18d4-4cfa-94fd-6f9af33f2acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:18.405086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca7582-18d4-4cfa-94fd-6f9af33f2acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:33:18.406510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca7582-18d4-4cfa-94fd-6f9af33f2acb service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:33:18.417848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1155/2304] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:33:18.704473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:33:18.705930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:18.719035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924888fc-161f-49f7-96db-6eefb1f87a9a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:33:18.724809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:33:18.725526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:18.725897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:18.726239 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] PATH_INFO: `/v3/projects/6142fb9ed8a3428ca2216368d073a13c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:18.727446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6142fb9ed8a3428ca2216368d073a13c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:33:18.729885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:33:18.779414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6142fb9ed8a3428ca2216368d073a13c', 'request_id': 'req-35633a3b-edf2-4f84-9bde-c126e545016b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.788140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6142fb9ed8a3428ca2216368d073a13c', 'request_id': 'req-35633a3b-edf2-4f84-9bde-c126e545016b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.794016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6142fb9ed8a3428ca2216368d073a13c', 'request_id': 'req-35633a3b-edf2-4f84-9bde-c126e545016b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.805366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] The token cache is being invalidate because project 6142fb9ed8a3428ca2216368d073a13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:33:18.805650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.806284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-35633a3b-edf2-4f84-9bde-c126e545016b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:33:18.807810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1150/2305] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:33:18 2022] DELETE /identity/v3/projects/6142fb9ed8a3428ca2216368d073a13c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:33:20.773524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dd032-6bd7-48bd-af7a-778c30005edb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:20.774021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dd032-6bd7-48bd-af7a-778c30005edb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:20.774021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dd032-6bd7-48bd-af7a-778c30005edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:20.793764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c66dd032-6bd7-48bd-af7a-778c30005edb None None] MFA Rules not processed for user `4659e9d9c65e436aa101db62a801a5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:33:20.794957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c66dd032-6bd7-48bd-af7a-778c30005edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:20.813587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:33:20.859176 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1156/2306] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:33:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:33:28.802530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:33:30.852501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=15 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:33:53.280454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feec448-61db-41d4-9697-945eb929ae50 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:33:53.280972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feec448-61db-41d4-9697-945eb929ae50 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:33:53.281076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feec448-61db-41d4-9697-945eb929ae50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:33:53.301119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2feec448-61db-41d4-9697-945eb929ae50 None None] MFA Rules not processed for user `3e2bdc73390a43ff9eccc57948e2b4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:33:53.302334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2feec448-61db-41d4-9697-945eb929ae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:33:53.366975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1151/2307] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:33:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:00.823092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:00.823680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:00.823797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:00.844323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] MFA Rules not processed for user `0a6757e5408e40cc85ad6145064d8ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:00.845623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:00.904008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1157/2308] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:03.360009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:10.897043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=25 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:22.049953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:22.050459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:22.050548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:22.070380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] MFA Rules not processed for user `472b6623f57a41568fe7ffce8e40e350`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:22.071631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:22.127568 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1152/2309] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:22.758285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:22.759723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:22.825168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bd03a-604e-4142-83e5-ca30e662bb6e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:22.832726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:22.834087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:22.834281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:22.834562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:22.836285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:22.923481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:22.924963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf0a71-3781-4a77-92b3-33f5176ef0da service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:22.936511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1158/2310] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:26.338694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:26.340212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:26.416109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b39e-87a0-4db8-a5f5-8d9e269a156e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:26.421051 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:26.421733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:26.422045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:26.422561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:26.424503 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:26.440963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:26.443342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40bf81-d760-4ea8-8d17-c639a038979b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:26.456421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1153/2311] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:28.425404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacff06-3ffb-4be1-91de-e633acf4a71d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:28.425978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacff06-3ffb-4be1-91de-e633acf4a71d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:28.426083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacff06-3ffb-4be1-91de-e633acf4a71d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:28.428126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1159/2312] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:34:28.461017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:28.461563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:28.461563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:28.483780 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:28.485053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:28.560465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1154/2313] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:34:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:28.561410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacff06-3ffb-4be1-91de-e633acf4a71d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:28.563300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ebacff06-3ffb-4be1-91de-e633acf4a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:28.586199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:28.589110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:28.602914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1289c-48e5-4bb9-9c16-b81589048137 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:28.608435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:28.609156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:28.609616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:28.609901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:28.618981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:28.620614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:28.631575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1155/2314] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:34:28.671784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacff06-3ffb-4be1-91de-e633acf4a71d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:28.676461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:28.677172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:28.677472 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:28.677730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] PATH_INFO: `/v3/users/4659e9d9c65e436aa101db62a801a5fe` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:28.678899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4659e9d9c65e436aa101db62a801a5fe)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:28.680865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:28.728549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:28.731532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4659e9d9c65e436aa101db62a801a5fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:28.739962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4659e9d9c65e436aa101db62a801a5fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:28.740596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4659e9d9c65e436aa101db62a801a5fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:28.741208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4659e9d9c65e436aa101db62a801a5fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:28.750219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-27bae17f-19ac-4dc7-9dda-4c432526e21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4659e9d9c65e436aa101db62a801a5fe'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:28.773422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1160/2315] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:34:28 2022] DELETE /identity/v3/users/4659e9d9c65e436aa101db62a801a5fe => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:34:28.960358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:28.971960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d49e-e174-4f06-8f6c-93311cca1be4 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.012189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:29.042787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.043434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.043750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.044026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] PATH_INFO: `/v3/projects/9c96a5a4edc84b70a31c6b0cd2a8f864` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.045212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c96a5a4edc84b70a31c6b0cd2a8f864)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.047174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.095453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c96a5a4edc84b70a31c6b0cd2a8f864', 'request_id': 'req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:29.104007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c96a5a4edc84b70a31c6b0cd2a8f864', 'request_id': 'req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:29.110042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c96a5a4edc84b70a31c6b0cd2a8f864', 'request_id': 'req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:29.121831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] The token cache is being invalidate because project 9c96a5a4edc84b70a31c6b0cd2a8f864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:34:29.122121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:29.122774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-18d169f5-f9a8-45f2-938e-cb6f6d64fd71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:29.124299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1156/2316] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:34:28 2022] DELETE /identity/v3/projects/9c96a5a4edc84b70a31c6b0cd2a8f864 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:34:29.129811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.130526 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.130984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.152953 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:29.154155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:29.220850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1161/2317] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:29.228759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.229042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.229315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.250335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:29.251602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:29.260277 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1157/2318] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:29.264565 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.274667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb346d9e-6706-45d4-9b4e-33da5e5563b0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.279196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.279847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.280119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.280388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.281027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:34:29.281323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.282898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.295130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1162/2319] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:34:29.299691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.309972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866daaa-c03b-4797-85b5-6056e649ee2c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.315134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.315866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.316215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.316560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.317810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.319822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.350791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1158/2320] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:34:29 2022] POST /identity/v3/projects => generated 395 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:34:29.355188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.366157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355055-569a-4a88-8416-ee5c127436ef admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.424837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.425600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.426259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.426375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.427603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.429469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.450266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1163/2321] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:34:29 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:34:29.455260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.466587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e9f2c-3fc0-4a86-b3f9-a69454f29355 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.471116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.471746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.472027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.472616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.473201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.475994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.486857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1159/2322] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:34:29.491522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.501685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a293-cceb-4062-ab2d-3b27e65bbd72 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.506464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d58227-8c5e-4a50-969c-0e2fc93884e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.507062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d58227-8c5e-4a50-969c-0e2fc93884e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.507340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d58227-8c5e-4a50-969c-0e2fc93884e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.507637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d58227-8c5e-4a50-969c-0e2fc93884e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.508312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d58227-8c5e-4a50-969c-0e2fc93884e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.509923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d58227-8c5e-4a50-969c-0e2fc93884e6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.520418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1164/2323] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:34:29.524784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.538454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f0108-9fd8-4ceb-a38e-df9fb6d96d2f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.544295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.544967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.545247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.545523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] PATH_INFO: `/v3/projects/e8fd480c2a994582b9450174f42668f2/users/eb1aa017161244bdbb7bce26d8afea64/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.561354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fd480c2a994582b9450174f42668f2, user_id=eb1aa017161244bdbb7bce26d8afea64, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.564170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.581153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1160/2324] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:34:29 2022] PUT /identity/v3/projects/e8fd480c2a994582b9450174f42668f2/users/eb1aa017161244bdbb7bce26d8afea64/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:34:29.592981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.593266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.593539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.622927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] MFA Rules not processed for user `eb1aa017161244bdbb7bce26d8afea64`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:29.624726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:29.639420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:29.700389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1165/2325] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:29.712519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.715729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:29.781006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0aa76-7e7f-449e-9bdf-f7c58a6de424 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.784607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.785242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.785517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.785787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.794592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.796067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.800054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:29.801142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:29.801427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:29.807262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1161/2326] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:29 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:29.873608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a269a-8436-420a-9ee4-01770b6c7a46 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:29.877962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:29.877962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:29.878232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:29.878416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:29.881225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:29.959442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:29.961001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e24d07-90d8-4a12-a9f2-4d9a3a32c65d service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:29.973931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1166/2327] 10.222.0.25 () {60 vars in 1396 bytes} [Fri Sep 9 06:34:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:34:37.075514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:37.142226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15491cf6-35bf-4202-af55-40430550689d service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:37.147597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:37.148291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:37.148669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:37.148895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] PATH_INFO: `/v3/users/eb1aa017161244bdbb7bce26d8afea64` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:37.150139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1aa017161244bdbb7bce26d8afea64)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:37.152268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:37.205498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1aa017161244bdbb7bce26d8afea64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.213383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1aa017161244bdbb7bce26d8afea64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.214601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1aa017161244bdbb7bce26d8afea64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.222292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1aa017161244bdbb7bce26d8afea64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.234201 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1aa017161244bdbb7bce26d8afea64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.240259 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1162/2328] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:34:37 2022] DELETE /identity/v3/users/eb1aa017161244bdbb7bce26d8afea64 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:34:37.244738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048811e-facb-432e-ad34-14770a069e30 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:37.245294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048811e-facb-432e-ad34-14770a069e30 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:37.245294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048811e-facb-432e-ad34-14770a069e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:37.264850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9048811e-facb-432e-ad34-14770a069e30 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:37.266091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9048811e-facb-432e-ad34-14770a069e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:37.328588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1167/2329] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:37.339125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:37.340755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:37.399512 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dcd7e-bb07-4b42-9205-2b55daeba287 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:37.403599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:37.404207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:37.404481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:37.404749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:37.414882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:37.416333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:37.425824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1163/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:37.557252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:37.557546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:37.558172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:37.579179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] MFA Rules not processed for user `925d5f23bd3f4b35a66736d2a6a3c3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:37.580430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:37.586452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:37.652836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1168/2331] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:37.747483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:37.749092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:37.764627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cf9a6-4ae6-4ff4-8568-633dd8bbcc2f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:37.781797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:37.782502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:37.783058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:37.783420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] PATH_INFO: `/v3/projects/e8fd480c2a994582b9450174f42668f2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:37.784931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fd480c2a994582b9450174f42668f2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:37.786959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:37.804843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:37.838462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fd480c2a994582b9450174f42668f2', 'request_id': 'req-631f59eb-9ebd-465e-9f71-b2d7b10933c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.847605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fd480c2a994582b9450174f42668f2', 'request_id': 'req-631f59eb-9ebd-465e-9f71-b2d7b10933c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.853792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fd480c2a994582b9450174f42668f2', 'request_id': 'req-631f59eb-9ebd-465e-9f71-b2d7b10933c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.866274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] The token cache is being invalidate because project e8fd480c2a994582b9450174f42668f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:34:37.866562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.867224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-631f59eb-9ebd-465e-9f71-b2d7b10933c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:34:37.868727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1164/2332] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:34:37 2022] DELETE /identity/v3/projects/e8fd480c2a994582b9450174f42668f2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:34:38.256577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:38.257961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:38.269562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4e51e-9a4c-4557-938f-9ae6534895eb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:38.317984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:38.318573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:38.318846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:38.319114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:38.320973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:38.329920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:38.331325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d54e-df20-445b-ae8e-20e7a60c8775 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:38.339631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1169/2333] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:38.651828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db19fab-62ae-4be7-b815-886af9bec587 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:38.652252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db19fab-62ae-4be7-b815-886af9bec587 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:38.652554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db19fab-62ae-4be7-b815-886af9bec587 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:38.653961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1165/2334] 10.222.0.25 () {54 vars in 961 bytes} [Fri Sep 9 06:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 06:34:38.659225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:38.659516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:38.659847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:38.681180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] MFA Rules not processed for user `31e2bef8256c40c68bc7711c5922289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:38.682313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:38.751259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1170/2335] 10.222.0.25 () {58 vars in 1058 bytes} [Fri Sep 9 06:34:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:38.758861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db19fab-62ae-4be7-b815-886af9bec587 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:38.772155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db19fab-62ae-4be7-b815-886af9bec587 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:38.777092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:38.777706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:38.778006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:38.778224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:38.780020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:38.792575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:38.794355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:38.805908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1166/2336] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:39.737344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:39.738752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:39.801085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9cfeb-31e0-420b-a9a5-38fbc8caa8b4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:39.804727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:39.805342 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:39.805620 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:39.805894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:39.807755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:39.817191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:39.873606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:39.875042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:39.885135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1171/2337] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:40.633496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:40.635233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:40.650094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140d31a-5d51-453e-8f23-8573a0ebd522 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:40.655612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:40.657173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:40.657628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:40.658474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:40.661270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:40.727837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:40.729378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:40.740659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1167/2338] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:43.247959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:43.249435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:43.261938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a3f3a-e6d4-4e44-a143-4125103862de service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:43.266199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:43.266786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:43.267065 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:43.267341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:43.269084 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:43.279963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:43.281397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e03d66-f55e-4267-8312-5da132178ddf service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:43.290177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1172/2339] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:49.833383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:49.835145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:50.023666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc101e47-df3c-444d-9b37-7e8221882405 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:50.032756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:50.033543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:50.033920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:50.034413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:50.036492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:50.054143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:50.055968 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:50.079961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1168/2340] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:52.153936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:52.154441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:52.154566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:52.175276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] MFA Rules not processed for user `1c86ccfec70340c797ecba83163ff1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:34:52.176628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:52.190215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1173/2341] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:34:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:34:55.993542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:55.995068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:56.009200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9917a-d55a-4f7f-a47d-37e6deeeaaa4 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:56.014844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:56.015527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:56.015843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:56.016125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:56.017846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:56.032757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:34:56.084572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:56.086176 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:56.095597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1169/2342] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:34:59.706096 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:34:59.749880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db31b01-b7a9-4f10-add5-5a60857baacf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:34:59.758447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:34:59.759488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:34:59.760021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:34:59.760685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:34:59.763810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:34:59.780192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:34:59.782359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:34:59.797254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1174/2343] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:34:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:06.091243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=21 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:06.891561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:06.892981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:06.904425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad26c-98fb-477b-a2e2-c2593627488b service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:06.907974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:06.908574 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:06.908859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:06.909134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:06.911269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:06.977282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:06.979283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:06.990388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1170/2344] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:09.791129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=38 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:10.549633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:10.551749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:10.566306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ea8c-4827-41d2-a7e7-589e36df20ac service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:10.571027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:10.571793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:10.572141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:10.572422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] PATH_INFO: `/v3/users/3e2bdc73390a43ff9eccc57948e2b4a6` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:10.573554 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2bdc73390a43ff9eccc57948e2b4a6)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:10.575475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:10.624299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2bdc73390a43ff9eccc57948e2b4a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.632085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2bdc73390a43ff9eccc57948e2b4a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.632691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2bdc73390a43ff9eccc57948e2b4a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.633292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2bdc73390a43ff9eccc57948e2b4a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.640322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6b1f4d1a-95a7-4b66-a19f-e472cf9fbe7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2bdc73390a43ff9eccc57948e2b4a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.657795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1175/2345] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:10 2022] DELETE /identity/v3/users/3e2bdc73390a43ff9eccc57948e2b4a6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:10.830418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:10.840789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73de7cb-5f4a-41f2-a740-e84244e26f1b service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:10.900604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:10.901248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:10.901533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:10.901805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] PATH_INFO: `/v3/projects/9fae936c35c54adfaf8cb0df7f4ca9dd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:10.902916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fae936c35c54adfaf8cb0df7f4ca9dd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:10.904837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:10.950124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fae936c35c54adfaf8cb0df7f4ca9dd', 'request_id': 'req-2c70ec92-2299-49a9-a2e0-caad364c20ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.957670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fae936c35c54adfaf8cb0df7f4ca9dd', 'request_id': 'req-2c70ec92-2299-49a9-a2e0-caad364c20ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.963306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fae936c35c54adfaf8cb0df7f4ca9dd', 'request_id': 'req-2c70ec92-2299-49a9-a2e0-caad364c20ba'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.975085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] The token cache is being invalidate because project 9fae936c35c54adfaf8cb0df7f4ca9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:10.975369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.976031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2c70ec92-2299-49a9-a2e0-caad364c20ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:10.977558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1171/2346] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:10 2022] DELETE /identity/v3/projects/9fae936c35c54adfaf8cb0df7f4ca9dd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:10.982424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:10.982879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:10.982973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.002816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:11.004094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:11.062759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1176/2347] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:11.070600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.070886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.071163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.090869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:11.092105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:11.100986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1172/2348] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:11.105087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.115089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ed679-117e-4f35-83a0-8e9bc80a6044 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.119654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.120252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.120530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.120794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.121432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:11.121733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.123316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.134899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1177/2349] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:11.139123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.148834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e2cc1-a04d-497d-a2b4-ba1d188f984d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.153427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.154021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.154300 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.154570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.155195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.157112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.173161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:11.186225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1173/2350] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:11 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:11.190282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.200411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12c3be-257c-43fd-972d-a265b274bcab admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.257016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.257612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.257884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.258161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.258780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.260709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.268498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:11.283727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1178/2351] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:11 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:35:11.288797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.298755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f05f09-ff02-44cb-a9a8-330fb6635083 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.303532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.304162 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.304439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.304717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.305364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.306957 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.316916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1174/2352] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:11.320680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.330577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb968f-ee9b-4395-8946-f9bb0f8d7917 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.335705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae47316-5770-4bb7-a04c-4d99dbf7f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.336299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47316-5770-4bb7-a04c-4d99dbf7f84d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.336579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47316-5770-4bb7-a04c-4d99dbf7f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.336849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae47316-5770-4bb7-a04c-4d99dbf7f84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.337483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47316-5770-4bb7-a04c-4d99dbf7f84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.339052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae47316-5770-4bb7-a04c-4d99dbf7f84d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.351387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1179/2353] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:11.355511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.365705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b9fcd-051a-4326-ba43-4b5f4bc69f10 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.370217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.370857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.371137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.371412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] PATH_INFO: `/v3/projects/2f1a472904b84adf8b0d2f4534da907b/users/a5dff88c013349bcabc48914c0be6d16/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.385624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1a472904b84adf8b0d2f4534da907b, user_id=a5dff88c013349bcabc48914c0be6d16, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.388295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.404359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1175/2354] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:11 2022] PUT /identity/v3/projects/2f1a472904b84adf8b0d2f4534da907b/users/a5dff88c013349bcabc48914c0be6d16/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:11.417217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.417504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.418192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.449640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] MFA Rules not processed for user `a5dff88c013349bcabc48914c0be6d16`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:11.450891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:11.521205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1180/2355] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:11.532754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:11.534694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:11.623956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df266-b429-42b4-996f-a1b61a72f20f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:11.630381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97240108-100a-424d-ae10-af1612f91cfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:11.630995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240108-100a-424d-ae10-af1612f91cfa service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:11.631276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240108-100a-424d-ae10-af1612f91cfa service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:11.632002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240108-100a-424d-ae10-af1612f91cfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:11.639663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97240108-100a-424d-ae10-af1612f91cfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:11.641414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97240108-100a-424d-ae10-af1612f91cfa service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:11.651213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1176/2356] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:14.324106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:14.384544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ceaf1-f951-483f-9995-d06d6466c0f6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:14.388969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:14.389594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:14.389878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:14.390145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] PATH_INFO: `/v3/users/a5dff88c013349bcabc48914c0be6d16` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:14.391267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5dff88c013349bcabc48914c0be6d16)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:14.393218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:14.439050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5dff88c013349bcabc48914c0be6d16'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:14.446567 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5dff88c013349bcabc48914c0be6d16'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:14.447183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5dff88c013349bcabc48914c0be6d16'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:14.447810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5dff88c013349bcabc48914c0be6d16'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:14.454584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5dff88c013349bcabc48914c0be6d16'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:14.460066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:14.475570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1181/2357] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:14 2022] DELETE /identity/v3/users/a5dff88c013349bcabc48914c0be6d16 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:14.480146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:14.480593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:14.480690 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:14.500479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:14.501670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:14.563100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1177/2358] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:14.572984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:14.574399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:14.631044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e550-7a1c-4954-85c7-e9bad2adb643 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:14.634628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4f2eb-38fb-402f-ac5d-346a3808f026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:14.635170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4f2eb-38fb-402f-ac5d-346a3808f026 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:14.635448 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4f2eb-38fb-402f-ac5d-346a3808f026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:14.635758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4f2eb-38fb-402f-ac5d-346a3808f026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:14.644782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4f2eb-38fb-402f-ac5d-346a3808f026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:14.646202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4f2eb-38fb-402f-ac5d-346a3808f026 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:14.655893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1182/2359] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:14.925784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:14.927285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:14.939865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c711a74-093c-4744-b35d-0b237d692f99 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:14.944965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:14.946347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:14.946688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:14.946988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] PATH_INFO: `/v3/projects/2f1a472904b84adf8b0d2f4534da907b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:14.948183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1a472904b84adf8b0d2f4534da907b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:14.950075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:14.994260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:15.004197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1a472904b84adf8b0d2f4534da907b', 'request_id': 'req-d3b3edf9-3d14-4e61-9f26-3eefffae3481'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:15.012916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1a472904b84adf8b0d2f4534da907b', 'request_id': 'req-d3b3edf9-3d14-4e61-9f26-3eefffae3481'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:15.019856 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1a472904b84adf8b0d2f4534da907b', 'request_id': 'req-d3b3edf9-3d14-4e61-9f26-3eefffae3481'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:15.031751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] The token cache is being invalidate because project 2f1a472904b84adf8b0d2f4534da907b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:15.032101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:15.032761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d3b3edf9-3d14-4e61-9f26-3eefffae3481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:15.035208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1178/2360] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:14 2022] DELETE /identity/v3/projects/2f1a472904b84adf8b0d2f4534da907b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:15.040881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.042048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.042450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.062091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:15.063404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:15.131573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1183/2361] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:15.135894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.136211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.136487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.156309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:15.157506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:15.165908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1179/2362] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:15.170249 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.180558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496c1d8-c070-4f73-aff2-389276aa470b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.186133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.186692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.186982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.187386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.188301 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:15.188608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.190195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.202333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1184/2363] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:15.207658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.216830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615c5ba-196c-4add-b583-346a3ee0abbe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.221465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.222549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.222853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.223126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.223806 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.225706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.252739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1180/2364] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:15 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:15.257496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.267516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2839e-a15a-49df-ac0b-e85423ab3917 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.323819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.324418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.325010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.325010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.325549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.327444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.350667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1185/2365] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:15 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:15.354884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.365607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5cf64-ffaa-444b-96bf-643721a302a0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.370213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.370818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.371092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.371362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.372037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.373627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.384200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1181/2366] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:15.388023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.398491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94ed79-83a7-42b8-82dc-11def35ae9cc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.403055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59356cc-3248-442b-906d-0cd6462f8e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.403686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59356cc-3248-442b-906d-0cd6462f8e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.403975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59356cc-3248-442b-906d-0cd6462f8e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.404246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59356cc-3248-442b-906d-0cd6462f8e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.404888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59356cc-3248-442b-906d-0cd6462f8e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.406512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59356cc-3248-442b-906d-0cd6462f8e69 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.407732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:15.418189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1186/2367] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:15.422445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.432455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0a09e-4154-4b4d-8f3b-b312f6a30f0c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.437050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.437692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.437973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.438245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] PATH_INFO: `/v3/projects/a3524ff7c76540f8b89eed4860fb9d0d/users/c259d55f8cf541a1b27c8211684e5ca0/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.453374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3524ff7c76540f8b89eed4860fb9d0d, user_id=c259d55f8cf541a1b27c8211684e5ca0, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.456639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.470368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1182/2368] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:15 2022] PUT /identity/v3/projects/a3524ff7c76540f8b89eed4860fb9d0d/users/c259d55f8cf541a1b27c8211684e5ca0/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:15.480817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e703cee-c7b1-43b3-b219-27cb73579791 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.481101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e703cee-c7b1-43b3-b219-27cb73579791 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.481376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e703cee-c7b1-43b3-b219-27cb73579791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.509517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5e703cee-c7b1-43b3-b219-27cb73579791 None None] MFA Rules not processed for user `c259d55f8cf541a1b27c8211684e5ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:15.510786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e703cee-c7b1-43b3-b219-27cb73579791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:15.575871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1187/2369] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:15.586777 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:15.588499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:15.645435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf589aca-1bb0-40f6-a1fc-a8694bafffa1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:15.649020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:15.649641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:15.649926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:15.650194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:15.658465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:15.659858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:15.668470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1183/2370] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:23.238699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:23.239197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:23.239587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:23.261058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:23.262275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:23.321121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1188/2371] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:35:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:23.370116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:23.371532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:23.454797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa2a2-5994-4a55-a400-423306f8658f service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:23.460449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512784b4-75f3-4683-bfb9-31c1e6760447 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:23.461056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512784b4-75f3-4683-bfb9-31c1e6760447 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:23.461343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512784b4-75f3-4683-bfb9-31c1e6760447 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:23.461616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512784b4-75f3-4683-bfb9-31c1e6760447 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:23.469747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512784b4-75f3-4683-bfb9-31c1e6760447 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:23.471367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512784b4-75f3-4683-bfb9-31c1e6760447 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:23.475458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:23.481964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1184/2372] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:35:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:35:24.093596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:24.166197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4676296-004b-49b3-9c65-034b97ae0936 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:24.170724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:24.171373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:24.171664 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:24.171941 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] PATH_INFO: `/v3/users/c259d55f8cf541a1b27c8211684e5ca0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:24.173055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c259d55f8cf541a1b27c8211684e5ca0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:24.175285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:24.216127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:24.232475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c259d55f8cf541a1b27c8211684e5ca0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:24.241951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c259d55f8cf541a1b27c8211684e5ca0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:24.242853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c259d55f8cf541a1b27c8211684e5ca0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:24.243857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c259d55f8cf541a1b27c8211684e5ca0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:24.253939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c259d55f8cf541a1b27c8211684e5ca0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:24.279143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1189/2373] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:24 2022] DELETE /identity/v3/users/c259d55f8cf541a1b27c8211684e5ca0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:24.283492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:24.283955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:24.284454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:24.305971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:24.307349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:24.392502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1185/2374] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:24.403655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:24.405380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:24.460163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:24.463774 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:24.514581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b19fc-5282-4016-a8db-3f94635e04e7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:24.524190 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:24.524952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:24.525337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:24.525729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:24.539355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:24.541797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:24.560703 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1190/2375] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:24.622289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367df2-190a-49d3-b3a8-432419718200 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:24.629902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:24.630547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:24.630827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:24.631749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:24.633710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:24.651487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:24.653394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a130fd-a058-4b91-b9f6-af58f65cbfb2 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:24.666702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1186/2376] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:35:24.980783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:24.983066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:25.017198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6d0f8-5d60-4a9d-9f22-92ba2cd34b86 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.024083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.024729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.025006 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.025276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] PATH_INFO: `/v3/projects/a3524ff7c76540f8b89eed4860fb9d0d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.026397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3524ff7c76540f8b89eed4860fb9d0d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.028578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.084606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3524ff7c76540f8b89eed4860fb9d0d', 'request_id': 'req-3bbf5517-73cd-4744-b613-b081d478496e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:25.091710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3524ff7c76540f8b89eed4860fb9d0d', 'request_id': 'req-3bbf5517-73cd-4744-b613-b081d478496e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:25.101405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3524ff7c76540f8b89eed4860fb9d0d', 'request_id': 'req-3bbf5517-73cd-4744-b613-b081d478496e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:25.115759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] The token cache is being invalidate because project a3524ff7c76540f8b89eed4860fb9d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:25.116052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:25.116774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3bbf5517-73cd-4744-b613-b081d478496e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:25.118263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1191/2377] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:24 2022] DELETE /identity/v3/projects/a3524ff7c76540f8b89eed4860fb9d0d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:25.123298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.123631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.123921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.151496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:25.157096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:25.226721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1187/2378] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:25.235811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.236048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.236379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.257115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:25.259123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:25.267219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1192/2379] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:25.272559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.285236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ab7cd-e6b9-4667-b759-e36db0e9823b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.291264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.291935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.292297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.292491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.293151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:25.293465 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.295085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.311760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1188/2380] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:25.313100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.323581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c689631-d1fe-48a5-a44a-88c1dec38cfd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.332489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.333105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.333436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.333713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.334366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.336388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.368593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1193/2381] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:25 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:25.372782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.384236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727b769-5792-44d6-92a0-041ebb00fb5a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.395011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:25.453880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.454505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.454853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.455129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.456378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.458381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.483207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1189/2382] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:25 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:25.488370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.501307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87146108-d39d-4b86-bcff-a674a1bc7150 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.508377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.509027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.509283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.509562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.510218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.511855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.513334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:25.530735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1194/2383] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:25.535296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.553260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f583b-1451-4579-8b00-85913f175d2f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.558057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ac0ca-2c83-41e5-b84e-ae88c76d7f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.558723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac0ca-2c83-41e5-b84e-ae88c76d7f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.559010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac0ca-2c83-41e5-b84e-ae88c76d7f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.559282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac0ca-2c83-41e5-b84e-ae88c76d7f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.559960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ac0ca-2c83-41e5-b84e-ae88c76d7f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.561585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ac0ca-2c83-41e5-b84e-ae88c76d7f92 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.575590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1190/2384] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:25.579921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.591437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a09f9-758a-4083-80d0-9d7fb96b5bb0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.596636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.598045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.598332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.598608 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] PATH_INFO: `/v3/projects/1a5dd18d26dc489386fd67fd40ad04cd/users/acf2999b361f429e86201f6bf8a8d6ca/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.616623 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5dd18d26dc489386fd67fd40ad04cd, user_id=acf2999b361f429e86201f6bf8a8d6ca, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.619287 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.639497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1195/2385] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:25 2022] PUT /identity/v3/projects/1a5dd18d26dc489386fd67fd40ad04cd/users/acf2999b361f429e86201f6bf8a8d6ca/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:25.641133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5545a-a910-4ce0-93c0-35e4dd31839e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.641443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5545a-a910-4ce0-93c0-35e4dd31839e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.641722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5545a-a910-4ce0-93c0-35e4dd31839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.663950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c3d5545a-a910-4ce0-93c0-35e4dd31839e None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:25.665284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c3d5545a-a910-4ce0-93c0-35e4dd31839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:25.735795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1191/2386] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:25.743995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:25.746142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:25.825633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a84e0-480a-485f-9d2c-0764db4283cf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:25.831240 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:25.831911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:25.832195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:25.832645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:25.843573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:25.845037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:25.868167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1196/2387] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:26.899528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:26.901045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:26.901508 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:26.955068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] MFA Rules not processed for user `acf2999b361f429e86201f6bf8a8d6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:26.956437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:27.070294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1192/2388] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:27.076170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:27.088304 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd02f43-4fb0-443c-9c85-00d04be84efe service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:27.093359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:27.094246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:27.094712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:27.095193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:27.096092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:27.098626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:27.141944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1197/2389] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:27 2022] POST /identity/v3/projects => generated 405 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:27.146087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:27.158150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2e3c7-5c41-4734-91ab-1c40cef375be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:27.187263 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:27.251094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:27.251931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:27.252317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:27.253244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:27.254210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:27.256340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:27.321600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1193/2390] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:27 2022] POST /identity/v3/users => generated 309 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:35:27.325442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:27.339435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b2ee-76fc-4358-ab47-b37d15e16afa admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:27.346009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:27.346009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:27.347199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:27.347539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:27.348865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:27.350147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:27.366338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1198/2391] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:35:27.370419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:27.389745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7d7af-8eca-429a-8edc-54c07573cc08 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:27.394880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:27.395736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:27.396127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:27.396510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] PATH_INFO: `/v3/projects/e156b2e029964c28bf5866dac7cc4121/users/2cb7f5fef2604a178c6314610cb040a9/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:27.416227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e156b2e029964c28bf5866dac7cc4121, user_id=2cb7f5fef2604a178c6314610cb040a9, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:27.419032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:27.435041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1194/2392] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:27 2022] PUT /identity/v3/projects/e156b2e029964c28bf5866dac7cc4121/users/2cb7f5fef2604a178c6314610cb040a9/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:27.439416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:27.451040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852e6f-cb99-496a-94d7-e3e0a5a709d5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:27.517284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20317d09-689a-4284-bac7-e90ea372443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:27.518726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317d09-689a-4284-bac7-e90ea372443d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:27.519100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317d09-689a-4284-bac7-e90ea372443d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:27.519577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20317d09-689a-4284-bac7-e90ea372443d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:27.520399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20317d09-689a-4284-bac7-e90ea372443d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:27.523173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20317d09-689a-4284-bac7-e90ea372443d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:27.538708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1199/2393] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:27.540429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:27.552638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b5bf3-fe43-461d-b7d4-3b8d84aebd04 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:27.557949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:27.558620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:27.558905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:27.559187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] PATH_INFO: `/v3/projects/e156b2e029964c28bf5866dac7cc4121/users/2cb7f5fef2604a178c6314610cb040a9/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:27.561684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e156b2e029964c28bf5866dac7cc4121, user_id=2cb7f5fef2604a178c6314610cb040a9, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:27.564398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:27.577585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1195/2394] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:27 2022] PUT /identity/v3/projects/e156b2e029964c28bf5866dac7cc4121/users/2cb7f5fef2604a178c6314610cb040a9/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:28.378284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:28.379203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:28.379203 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:28.445761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] MFA Rules not processed for user `2cb7f5fef2604a178c6314610cb040a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:28.447090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:28.495649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:28.602822 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1200/2395] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:29.907747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:29.999172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ff2ad-06b7-4278-a06c-88541770ef50 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:30.005956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:30.008691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:30.009163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:30.009894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] PATH_INFO: `/v3/users/acf2999b361f429e86201f6bf8a8d6ca` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:30.011183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf2999b361f429e86201f6bf8a8d6ca)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:30.013380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:30.072008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf2999b361f429e86201f6bf8a8d6ca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.084172 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf2999b361f429e86201f6bf8a8d6ca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.084833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf2999b361f429e86201f6bf8a8d6ca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.093095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf2999b361f429e86201f6bf8a8d6ca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.103100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:30.109148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf2999b361f429e86201f6bf8a8d6ca'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.121084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1196/2396] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:29 2022] DELETE /identity/v3/users/acf2999b361f429e86201f6bf8a8d6ca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:30.336759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:30.353959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f886ba-473b-4c86-8f41-07f89cc7f692 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:30.428918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:30.429601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:30.429882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:30.430152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] PATH_INFO: `/v3/projects/1a5dd18d26dc489386fd67fd40ad04cd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:30.431503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5dd18d26dc489386fd67fd40ad04cd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:30.433439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:30.491101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5dd18d26dc489386fd67fd40ad04cd', 'request_id': 'req-1742900b-029a-498f-8bdb-c92bcffbfdff'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.497760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5dd18d26dc489386fd67fd40ad04cd', 'request_id': 'req-1742900b-029a-498f-8bdb-c92bcffbfdff'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.509241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5dd18d26dc489386fd67fd40ad04cd', 'request_id': 'req-1742900b-029a-498f-8bdb-c92bcffbfdff'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.525140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] The token cache is being invalidate because project 1a5dd18d26dc489386fd67fd40ad04cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:30.525422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.526101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.527615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1201/2397] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:30 2022] DELETE /identity/v3/projects/1a5dd18d26dc489386fd67fd40ad04cd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:30.531736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:30.533168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:30.551020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2a14b-4395-4eb8-bb84-86d0643941d9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:30.622768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:30.623430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:30.623760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:30.624045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] PATH_INFO: `/v3/users/2cb7f5fef2604a178c6314610cb040a9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:30.625310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb7f5fef2604a178c6314610cb040a9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:30.627670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:30.685635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb7f5fef2604a178c6314610cb040a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.698076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb7f5fef2604a178c6314610cb040a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.698974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb7f5fef2604a178c6314610cb040a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.712255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb7f5fef2604a178c6314610cb040a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.727820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9c85a99c-94f5-43b7-b3c7-c4bd1369dcd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb7f5fef2604a178c6314610cb040a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:30.734213 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1197/2398] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:30 2022] DELETE /identity/v3/users/2cb7f5fef2604a178c6314610cb040a9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:30.974160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:30.976346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:31.005014 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742900b-029a-498f-8bdb-c92bcffbfdff admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.073549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:31.085055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.085674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.086575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.086855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] PATH_INFO: `/v3/projects/e156b2e029964c28bf5866dac7cc4121` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.088081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e156b2e029964c28bf5866dac7cc4121)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.090194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.146364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e156b2e029964c28bf5866dac7cc4121', 'request_id': 'req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:31.154631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e156b2e029964c28bf5866dac7cc4121', 'request_id': 'req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:31.165340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e156b2e029964c28bf5866dac7cc4121', 'request_id': 'req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:31.179123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] The token cache is being invalidate because project e156b2e029964c28bf5866dac7cc4121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:31.179461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:31.180145 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-51bc83ce-a4a8-4cca-856e-8d0e61ae8838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:31.181675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1202/2399] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:30 2022] DELETE /identity/v3/projects/e156b2e029964c28bf5866dac7cc4121 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:31.188275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.188825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.189200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.213868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:31.215252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:31.291464 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1198/2400] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:31.300571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931cb1-a238-4259-8098-066726be9760 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.301358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931cb1-a238-4259-8098-066726be9760 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.301912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931cb1-a238-4259-8098-066726be9760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.324094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-97931cb1-a238-4259-8098-066726be9760 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:31.325425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-97931cb1-a238-4259-8098-066726be9760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:31.340258 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1203/2401] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:31.341506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.352526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f64d60-8e95-430d-a5a9-8b6aa4ddc3ed None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.358102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.358807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.359325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.359959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.360543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:31.360951 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.362798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.370477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:31.384628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1199/2402] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:31.385862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97931cb1-a238-4259-8098-066726be9760 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.396719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97931cb1-a238-4259-8098-066726be9760 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.401730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.402510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.402916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.403305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.404076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.406086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.445408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1204/2403] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:31.448661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.460943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab57f0-0d1d-4d0a-ac9f-b3bcdc1d9d8f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.529069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.529705 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.529992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.530271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.530918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.532905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.558842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1200/2404] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:31.562750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.574332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f508-df94-4326-9712-815a40c78e26 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.582675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.583382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.583848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.584198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.584927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.586889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.600048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1205/2405] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:31.603681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.614074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5aa5c8-1543-4f8f-bdd8-ee26c3079fbe admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.619822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875bd66-461c-4fb3-8bbe-3acd7c48d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.620448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bd66-461c-4fb3-8bbe-3acd7c48d16f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.620731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bd66-461c-4fb3-8bbe-3acd7c48d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.621006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bd66-461c-4fb3-8bbe-3acd7c48d16f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.621668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875bd66-461c-4fb3-8bbe-3acd7c48d16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.623261 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875bd66-461c-4fb3-8bbe-3acd7c48d16f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.634988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1201/2406] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:31.639113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.650681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6d1f3-c8d7-4103-af59-c9841d5f6402 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.655486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.656195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.656477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.656757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] PATH_INFO: `/v3/projects/654127d3fb084898b42ae1ee2474de81/users/bb8e8c64b8594492ada7b2d4784f8d6f/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.674706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654127d3fb084898b42ae1ee2474de81, user_id=bb8e8c64b8594492ada7b2d4784f8d6f, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.677918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.697931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1206/2407] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:31 2022] PUT /identity/v3/projects/654127d3fb084898b42ae1ee2474de81/users/bb8e8c64b8594492ada7b2d4784f8d6f/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:31.709002 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.709531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.709986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.744714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] MFA Rules not processed for user `bb8e8c64b8594492ada7b2d4784f8d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:31.745982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:31.822389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1202/2408] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:31.826895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.897850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540225b-9fff-42bc-933f-81ba1b6674a4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:31.902831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:31.903455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:31.903780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:31.904060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:31.904707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:31.906686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:31.943094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1207/2409] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:31.947154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:31.960855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2c607-2784-4fd2-b69c-47aff2a09886 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.032519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.033226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.033647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.033991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.034773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.037088 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.051310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:32.065077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1203/2410] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:31 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:32.068579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:32.079030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:32.084445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bf375-0a84-4ca9-98b8-36b5eb2bc9ac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.090882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.091746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.092213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.092746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.093477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.095183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.112505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:32.112974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1208/2411] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:35:32.129714 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c3b5e-b67a-4239-8d6f-c2fad745466d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.134764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.135446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.135796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.136074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] PATH_INFO: `/v3/projects/259f43c3ae964a5ea60a359d07833ece/users/40e610172e2947c88215b12ae70ffc46/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.163488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] RBAC: Authorizing `identity:create_grant(project_id=259f43c3ae964a5ea60a359d07833ece, user_id=40e610172e2947c88215b12ae70ffc46, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.166534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.184005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1204/2412] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:32 2022] PUT /identity/v3/projects/259f43c3ae964a5ea60a359d07833ece/users/40e610172e2947c88215b12ae70ffc46/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:32.186978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:32.198407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3bc2e-6603-46da-8365-bca0717ed1be admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.292204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bf3e6-4458-494a-8129-368cdb2f6f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.292833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf3e6-4458-494a-8129-368cdb2f6f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.293119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf3e6-4458-494a-8129-368cdb2f6f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.293392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bf3e6-4458-494a-8129-368cdb2f6f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.294038 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bf3e6-4458-494a-8129-368cdb2f6f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.295734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bf3e6-4458-494a-8129-368cdb2f6f42 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.307590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1209/2413] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:32.311225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:32.324424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dbc20-2c88-4ba0-83a5-a3aaaeb1542d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.329434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.330042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.330935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.331238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] PATH_INFO: `/v3/projects/259f43c3ae964a5ea60a359d07833ece/users/40e610172e2947c88215b12ae70ffc46/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.333488 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259f43c3ae964a5ea60a359d07833ece, user_id=40e610172e2947c88215b12ae70ffc46, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.336216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.348753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1205/2414] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:32 2022] PUT /identity/v3/projects/259f43c3ae964a5ea60a359d07833ece/users/40e610172e2947c88215b12ae70ffc46/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:32.361085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db538be-953b-4083-9e58-add595b6a693 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.362185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db538be-953b-4083-9e58-add595b6a693 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.362185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db538be-953b-4083-9e58-add595b6a693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.394309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8db538be-953b-4083-9e58-add595b6a693 None None] MFA Rules not processed for user `40e610172e2947c88215b12ae70ffc46`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:32.395569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8db538be-953b-4083-9e58-add595b6a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:32.476311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1210/2415] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:32.488144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:32.489964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:32.581009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603a98-a4ba-44fd-b96b-58623741d036 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.584950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.585567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.585868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.586137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.595480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.597897 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.608049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1206/2416] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:32.661058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db538be-953b-4083-9e58-add595b6a693 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:32.662463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8db538be-953b-4083-9e58-add595b6a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:32.679101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db538be-953b-4083-9e58-add595b6a693 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:32.683235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e9dbe-def8-4960-8c13-9f2dcf28a286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:32.684229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e9dbe-def8-4960-8c13-9f2dcf28a286 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:32.684980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e9dbe-def8-4960-8c13-9f2dcf28a286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:32.685513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e9dbe-def8-4960-8c13-9f2dcf28a286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:32.755126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e9dbe-def8-4960-8c13-9f2dcf28a286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:32.756644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e9dbe-def8-4960-8c13-9f2dcf28a286 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:32.769505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1211/2417] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:36.135612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:36.196494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c205687-a778-434e-a016-a6afdb92c817 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:36.202648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:36.203282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:36.203585 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:36.203864 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] PATH_INFO: `/v3/users/bb8e8c64b8594492ada7b2d4784f8d6f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:36.204995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8e8c64b8594492ada7b2d4784f8d6f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:36.207100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:36.256312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8e8c64b8594492ada7b2d4784f8d6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.264165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8e8c64b8594492ada7b2d4784f8d6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.264982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8e8c64b8594492ada7b2d4784f8d6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.273050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8e8c64b8594492ada7b2d4784f8d6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.274206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:36.290433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8e8c64b8594492ada7b2d4784f8d6f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.296511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1207/2418] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:36 2022] DELETE /identity/v3/users/bb8e8c64b8594492ada7b2d4784f8d6f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:36.301083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:36.301617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:36.301849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:36.322681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:36.324659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:36.333635 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:36.388805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1212/2419] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:36.399431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:36.400948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:36.461416 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c38fdc-bb19-4e34-af96-64642b5da9d8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:36.465115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:36.465736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:36.466042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:36.466312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:36.475617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:36.477037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:36.486442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1208/2420] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:36.760319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:36.762192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:36.775992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0685-3c93-40ad-bf05-4b96fbc49494 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:36.781251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:36.782422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:36.782422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:36.782631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] PATH_INFO: `/v3/projects/654127d3fb084898b42ae1ee2474de81` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:36.783730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654127d3fb084898b42ae1ee2474de81)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:36.785626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:36.840720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654127d3fb084898b42ae1ee2474de81', 'request_id': 'req-fc553f39-8744-4c5c-9175-be587986c353'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.846445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654127d3fb084898b42ae1ee2474de81', 'request_id': 'req-fc553f39-8744-4c5c-9175-be587986c353'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.854758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654127d3fb084898b42ae1ee2474de81', 'request_id': 'req-fc553f39-8744-4c5c-9175-be587986c353'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.866913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] The token cache is being invalidate because project 654127d3fb084898b42ae1ee2474de81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:36.867202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.867999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:36.869413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1213/2421] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:36 2022] DELETE /identity/v3/projects/654127d3fb084898b42ae1ee2474de81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:36.873385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:36.883615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605073a1-6219-4a36-99f9-a15d04ce6b57 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:36.947139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:36.948443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:36.948728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:36.949056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] PATH_INFO: `/v3/users/40e610172e2947c88215b12ae70ffc46` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:36.950147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e610172e2947c88215b12ae70ffc46)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:36.952295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.005483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e610172e2947c88215b12ae70ffc46'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.013661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e610172e2947c88215b12ae70ffc46'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.014350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e610172e2947c88215b12ae70ffc46'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.022153 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e610172e2947c88215b12ae70ffc46'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.034410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1a604ba-f9f1-4885-88fb-c55086dc06b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e610172e2947c88215b12ae70ffc46'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.041077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1209/2422] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:36 2022] DELETE /identity/v3/users/40e610172e2947c88215b12ae70ffc46 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:37.325952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.327592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:37.345262 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc553f39-8744-4c5c-9175-be587986c353 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.412227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.412869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.413146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.413418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] PATH_INFO: `/v3/projects/259f43c3ae964a5ea60a359d07833ece` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.414543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259f43c3ae964a5ea60a359d07833ece)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.416501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.457172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:37.466516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259f43c3ae964a5ea60a359d07833ece', 'request_id': 'req-237084f6-2e03-4241-90c1-2d00a6f6daa2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.472851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259f43c3ae964a5ea60a359d07833ece', 'request_id': 'req-237084f6-2e03-4241-90c1-2d00a6f6daa2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.480745 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259f43c3ae964a5ea60a359d07833ece', 'request_id': 'req-237084f6-2e03-4241-90c1-2d00a6f6daa2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.493012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] The token cache is being invalidate because project 259f43c3ae964a5ea60a359d07833ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:37.493459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.494139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-237084f6-2e03-4241-90c1-2d00a6f6daa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:37.496105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1214/2423] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:37 2022] DELETE /identity/v3/projects/259f43c3ae964a5ea60a359d07833ece => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:37.501294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.501786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.501891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.509722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:37.523163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:37.525194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:37.591571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1210/2424] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:37.600606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.600891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.601164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.620970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:37.622340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:37.631127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1215/2425] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:37.635495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.645817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b17ed-3ae1-46f3-83f5-e7837d9dfc43 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.650912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.651529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.651833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.652102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.652757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:37.653131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.656065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.668824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1211/2426] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:37.673155 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.682536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093e17a-1c74-4c07-925d-87f8723829ab None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.687305 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.687951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.688245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.688498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.689141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.691629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.721484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1216/2427] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:37 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:35:37.725884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.735717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a95aa-6dd2-4c9c-9968-4bf51ab74ac6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.793645 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.794289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.794564 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.794832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.795454 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.797401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.819861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1212/2428] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:37 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:37.823995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.840675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfffba-2309-4bae-a563-93ec5421fd99 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.845583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.847437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.848414 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.848708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.849388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.851007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.863607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1217/2429] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:37.867275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.878524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037d5c-d011-4df4-be13-cea5b4211afa admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.889272 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdcadc-dc75-4cd4-ab4f-a4bfabd1d49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.889904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdcadc-dc75-4cd4-ab4f-a4bfabd1d49f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.890184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdcadc-dc75-4cd4-ab4f-a4bfabd1d49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.890455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdcadc-dc75-4cd4-ab4f-a4bfabd1d49f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.891111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdcadc-dc75-4cd4-ab4f-a4bfabd1d49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.893295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdcadc-dc75-4cd4-ab4f-a4bfabd1d49f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.905701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1213/2430] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:37.910580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:37.922468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32707006-3f8f-47d9-8331-af0c463df50b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:37.922981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2a9c3-ec8e-42fe-a43c-bddbe30a9cff None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.923327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2a9c3-ec8e-42fe-a43c-bddbe30a9cff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.923636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2a9c3-ec8e-42fe-a43c-bddbe30a9cff None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.926924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1214/2431] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:35:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:35:37.928899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3408076-4c36-4a53-be40-c64c91a2a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:37.929587 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3408076-4c36-4a53-be40-c64c91a2a389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.929863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3408076-4c36-4a53-be40-c64c91a2a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.930135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3408076-4c36-4a53-be40-c64c91a2a389 admin admin] PATH_INFO: `/v3/projects/6b5be53072624655aabba50458dbd594/users/c551d43112f04d489ef29f021cae904b/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.936240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.936599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.936939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:37.946590 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3408076-4c36-4a53-be40-c64c91a2a389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5be53072624655aabba50458dbd594, user_id=c551d43112f04d489ef29f021cae904b, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:37.949528 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3408076-4c36-4a53-be40-c64c91a2a389 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:37.961986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:37.963193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:37.964711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1218/2432] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:37 2022] PUT /identity/v3/projects/6b5be53072624655aabba50458dbd594/users/c551d43112f04d489ef29f021cae904b/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:37.976284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:37.976573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:37.976903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.006562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] MFA Rules not processed for user `c551d43112f04d489ef29f021cae904b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:38.007928 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.034577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:38.051436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1215/2433] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:35:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:38.068649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.070090 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.082355 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1219/2434] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:38.087040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.132861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced52530-f749-4bd8-a265-b0d975b720d2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.136474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.137081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.137352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.137617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.142389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:38.149104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.150708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.156232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93489de4-96ec-42c5-9b04-f5590f90b094 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.162127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d507c-fca2-491a-bd9d-c093aa015807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.162547 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1216/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:38.162761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d507c-fca2-491a-bd9d-c093aa015807 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.163022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d507c-fca2-491a-bd9d-c093aa015807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.163295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d507c-fca2-491a-bd9d-c093aa015807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.163959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d507c-fca2-491a-bd9d-c093aa015807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.165889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d507c-fca2-491a-bd9d-c093aa015807 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.201045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1220/2436] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:38 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:35:38.202483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.214489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c60b-05c3-4481-8a86-215558ae90a1 service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.233322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc92b-3859-4314-81ec-d93258e1a55b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.233612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc92b-3859-4314-81ec-d93258e1a55b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.233895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc92b-3859-4314-81ec-d93258e1a55b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.235238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1221/2437] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:35:38.240025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.240311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.241002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.260944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] MFA Rules not processed for user `86cd9a25b7ed4206a9b5939024168475`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:38.262226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.274381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.274914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.275222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.275481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.276155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.278101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.299935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1217/2438] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:38 2022] POST /identity/v3/users => generated 306 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:38.304105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.314161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa113615-58eb-4dcc-b137-bcf971ab3228 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.316562 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1222/2439] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:35:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:38.318974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.319616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.319903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.320184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.320830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.322423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.329241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.331450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.336456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1218/2440] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:38.337306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.342223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b1640-a04b-4f19-af1a-39441326857e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.346002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.347151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.347428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.347577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1eabd-59d4-49bb-9691-3436cf6dd574 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.347725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.352522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe7147-079f-4a03-93d8-e3a8440f2b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.353276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe7147-079f-4a03-93d8-e3a8440f2b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.353664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe7147-079f-4a03-93d8-e3a8440f2b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.354229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe7147-079f-4a03-93d8-e3a8440f2b3b admin admin] PATH_INFO: `/v3/projects/a8ee13773dbc48deb984fa0248422c90/users/21086ffb3a644d328bb26e2803ec8ef2/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.355095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.356666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.365415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1223/2441] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:35:38.370225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe7147-079f-4a03-93d8-e3a8440f2b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ee13773dbc48deb984fa0248422c90, user_id=21086ffb3a644d328bb26e2803ec8ef2, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.372915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe7147-079f-4a03-93d8-e3a8440f2b3b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.391132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1219/2442] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:38 2022] PUT /identity/v3/projects/a8ee13773dbc48deb984fa0248422c90/users/21086ffb3a644d328bb26e2803ec8ef2/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:38.394284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.415894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681f305-28e6-4ca9-acf4-a10447405e22 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.461054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79bd49-b66a-4345-b28c-fe1f5af0a3e5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.461613 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79bd49-b66a-4345-b28c-fe1f5af0a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.462220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79bd49-b66a-4345-b28c-fe1f5af0a3e5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.463775 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1220/2443] 10.222.0.25 () {54 vars in 921 bytes} [Fri Sep 9 06:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 06:35:38.469971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.470490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.471496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.490452 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.491069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.491351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.492027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.492298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.493645 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] MFA Rules not processed for user `dc56896d10094aed91db84ea2bf9dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:38.494251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.495017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.508605 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1224/2444] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:38.509570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.522109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248d826-15f7-4a21-8309-cf704d38a57a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.529071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b3216-1e99-4ad9-af2e-11a29340e498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.529674 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b3216-1e99-4ad9-af2e-11a29340e498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.530130 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b3216-1e99-4ad9-af2e-11a29340e498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.530475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b3216-1e99-4ad9-af2e-11a29340e498 admin admin] PATH_INFO: `/v3/projects/a8ee13773dbc48deb984fa0248422c90/users/21086ffb3a644d328bb26e2803ec8ef2/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.533411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b3216-1e99-4ad9-af2e-11a29340e498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ee13773dbc48deb984fa0248422c90, user_id=21086ffb3a644d328bb26e2803ec8ef2, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.537309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b3216-1e99-4ad9-af2e-11a29340e498 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.552959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1225/2445] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:38 2022] PUT /identity/v3/projects/a8ee13773dbc48deb984fa0248422c90/users/21086ffb3a644d328bb26e2803ec8ef2/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:38.566431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.567191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:38.567671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.568138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.590619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1221/2446] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Sep 9 06:35:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:38.605502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.605688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] MFA Rules not processed for user `21086ffb3a644d328bb26e2803ec8ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:38.606754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.606895 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.652124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:38.668411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f6e8-8b02-4612-914d-5b7144731735 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.678549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1226/2447] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:38.689340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.690984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:38.718470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a5fc-63ea-4504-b124-57fd28fcf4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.719082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a5fc-63ea-4504-b124-57fd28fcf4ac service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.719364 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a5fc-63ea-4504-b124-57fd28fcf4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.719676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a5fc-63ea-4504-b124-57fd28fcf4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.749564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d66c0-136e-4db5-967b-d250dea0cb96 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.753767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.754377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.754655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.754939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.763999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.765443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.776468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1227/2448] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:35:38.789468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1a5fc-63ea-4504-b124-57fd28fcf4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.790992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1a5fc-63ea-4504-b124-57fd28fcf4ac service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.804340 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1222/2449] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:35:38.855308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:38.866652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0701f-e5eb-47bc-afb5-46480e0349f3 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:38.870599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:38.871205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:38.871486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:38.871790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:38.929461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:38.930913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:38.941028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1228/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:40.643603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:40.644106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:40.644375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:40.664576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:40.665703 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:40.708091 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:40.722247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1223/2451] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:35:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:40.734211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:40.735606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:40.805046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaf981-33a3-4c20-b6ee-00df0c4e88c6 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:40.809959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c92384-7089-4380-ab26-bbe436bf322d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:40.810558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92384-7089-4380-ab26-bbe436bf322d service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:40.810838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92384-7089-4380-ab26-bbe436bf322d service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:40.811112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92384-7089-4380-ab26-bbe436bf322d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:40.819662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c92384-7089-4380-ab26-bbe436bf322d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:40.821247 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c92384-7089-4380-ab26-bbe436bf322d service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:40.829275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1229/2452] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:35:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:35:43.119912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:43.191108 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfec762-999e-46fb-b292-389fa3dff802 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:43.196012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:43.196660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:43.196935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:43.197200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] PATH_INFO: `/v3/users/c551d43112f04d489ef29f021cae904b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:43.208992 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c551d43112f04d489ef29f021cae904b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:43.211657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:43.265235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c551d43112f04d489ef29f021cae904b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:43.274161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c551d43112f04d489ef29f021cae904b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:43.274793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c551d43112f04d489ef29f021cae904b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:43.283281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c551d43112f04d489ef29f021cae904b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:43.296367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c551d43112f04d489ef29f021cae904b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:43.303868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1224/2453] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:43 2022] DELETE /identity/v3/users/c551d43112f04d489ef29f021cae904b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:43.308578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:43.309109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:43.309213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:43.343756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:43.345284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:43.354912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:43.440075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1230/2454] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:43.450335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:43.451928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:43.537221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15d78c-72d7-4d27-9332-536a730f365d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:43.541637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:43.542291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:43.542622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:43.542838 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:43.560173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:43.562168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:43.579044 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1225/2455] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:43.928869 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:43.930413 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:43.960676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc77afb-16cc-47ff-9de6-ef4993807637 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:43.973499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:43.974180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:43.975090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:43.975377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] PATH_INFO: `/v3/projects/6b5be53072624655aabba50458dbd594` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:43.984984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5be53072624655aabba50458dbd594)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:43.987442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:44.061774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5be53072624655aabba50458dbd594', 'request_id': 'req-635f75f7-037c-4c94-be32-c7a371d9ae10'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.071583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5be53072624655aabba50458dbd594', 'request_id': 'req-635f75f7-037c-4c94-be32-c7a371d9ae10'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.084044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5be53072624655aabba50458dbd594', 'request_id': 'req-635f75f7-037c-4c94-be32-c7a371d9ae10'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.112858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] The token cache is being invalidate because project 6b5be53072624655aabba50458dbd594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:44.113236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.113829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.116359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1231/2456] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:43 2022] DELETE /identity/v3/projects/6b5be53072624655aabba50458dbd594 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:44.119939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:44.132419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441561df-3c6a-4487-af5b-51dfe6e0b308 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:44.146022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:44.226056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:44.226713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:44.226996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:44.227275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] PATH_INFO: `/v3/users/21086ffb3a644d328bb26e2803ec8ef2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:44.228453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21086ffb3a644d328bb26e2803ec8ef2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:44.230385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:44.282276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21086ffb3a644d328bb26e2803ec8ef2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.291143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21086ffb3a644d328bb26e2803ec8ef2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.291799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21086ffb3a644d328bb26e2803ec8ef2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.299918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21086ffb3a644d328bb26e2803ec8ef2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.313150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9fad7a37-f6aa-4c78-a1c1-ffec10667227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21086ffb3a644d328bb26e2803ec8ef2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.319676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1226/2457] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:44 2022] DELETE /identity/v3/users/21086ffb3a644d328bb26e2803ec8ef2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:44.628231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:44.629719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:44.643675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f75f7-037c-4c94-be32-c7a371d9ae10 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:44.705858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:44.706600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:44.706894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:44.707165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] PATH_INFO: `/v3/projects/a8ee13773dbc48deb984fa0248422c90` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:44.708368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ee13773dbc48deb984fa0248422c90)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:44.710283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:44.760390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:44.770873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ee13773dbc48deb984fa0248422c90', 'request_id': 'req-b70f4da0-ad38-40ae-b4cb-21fe85f83617'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.780645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ee13773dbc48deb984fa0248422c90', 'request_id': 'req-b70f4da0-ad38-40ae-b4cb-21fe85f83617'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.790289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ee13773dbc48deb984fa0248422c90', 'request_id': 'req-b70f4da0-ad38-40ae-b4cb-21fe85f83617'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.805334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] The token cache is being invalidate because project a8ee13773dbc48deb984fa0248422c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:44.805710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.806580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b70f4da0-ad38-40ae-b4cb-21fe85f83617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:44.812022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1232/2458] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:44 2022] DELETE /identity/v3/projects/a8ee13773dbc48deb984fa0248422c90 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:44.813240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:44.813533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:44.813807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:44.856035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:44.857115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:44.950747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1227/2459] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:35:44.960083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:44.960371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:44.960716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:44.984119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:44.985509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:44.995595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1233/2460] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:44.998805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.008885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb961185-f6a4-44ab-9e2a-93ae5ed3cae6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.013478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.014079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.014359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.014641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.015304 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:45.015633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.017248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.030419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1228/2461] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:45.035840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.047572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10a889-a29a-4319-8cf4-601d8e59d5a7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.052711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.053335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.053615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.053888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.054530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.056675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.099334 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1234/2462] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:45 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:45.100167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.110963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c0c73-7d32-45c3-8a97-da5be90cd01b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.130369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:45.173792 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.174583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.175006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.175378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.176147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.178185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.221627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1229/2463] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:45 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:45.226076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.244069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ba2e6-1ef8-469c-931c-c3c2a1ab3cf1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.249850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.250543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.250906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.251241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.252100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.254114 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.264844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1235/2464] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:45.269050 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.279323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79538a78-7c14-4a17-a461-24817df7785b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.284666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be4ff-d1eb-43d6-a3f1-ab0e92c7a811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.285286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be4ff-d1eb-43d6-a3f1-ab0e92c7a811 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.285572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be4ff-d1eb-43d6-a3f1-ab0e92c7a811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.285847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be4ff-d1eb-43d6-a3f1-ab0e92c7a811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.286500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be4ff-d1eb-43d6-a3f1-ab0e92c7a811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.288157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be4ff-d1eb-43d6-a3f1-ab0e92c7a811 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.302604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1230/2465] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:35:45.306714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.318214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c2ebe-370c-447a-b413-f406d1d78e60 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.323086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.323917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.324331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.324687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] PATH_INFO: `/v3/projects/bc0fc6eafc344b58b7fdb9cdcdc3a2e3/users/bc5867aa24b84c469d33dbc609e3779e/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.340874 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0fc6eafc344b58b7fdb9cdcdc3a2e3, user_id=bc5867aa24b84c469d33dbc609e3779e, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.343563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.360939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1236/2466] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:45 2022] PUT /identity/v3/projects/bc0fc6eafc344b58b7fdb9cdcdc3a2e3/users/bc5867aa24b84c469d33dbc609e3779e/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:45.371583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.371986 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.372357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.406030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] MFA Rules not processed for user `bc5867aa24b84c469d33dbc609e3779e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:45.407461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:45.485063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1231/2467] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:45.492356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.561001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a1e9f-e6f7-4bc8-96f9-049803bd24c3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.567609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.567609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.567609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.568015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.568349 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.570216 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.605639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1237/2468] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:45.607097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.619331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd25ad-f417-4080-a9f0-a69704c79db2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.682222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.682955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.683240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.683646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.684621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.687110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.717140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1232/2469] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:45 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:45.718476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.731093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83850667-7b11-450e-b0a4-8212e115c225 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.737079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.737631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.737913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.738191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.738839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.740477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.752787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1238/2470] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:35:45.757126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.767839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e79129-2506-4400-8761-c0f5043deffc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.772921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.773889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.774197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.774626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] PATH_INFO: `/v3/projects/37d99d3c3027483398b352198ecf68c6/users/54868cb051f844ad9b2d02ed0075e296/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.791320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d99d3c3027483398b352198ecf68c6, user_id=54868cb051f844ad9b2d02ed0075e296, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.794252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.796643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:45.812640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1233/2471] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:45 2022] PUT /identity/v3/projects/37d99d3c3027483398b352198ecf68c6/users/54868cb051f844ad9b2d02ed0075e296/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:45.817709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.827889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed88bb-0fb0-4545-a2c9-41cd67ee83b3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.867264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:45.897691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0a8e-9432-4043-8793-01737915a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.898309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0a8e-9432-4043-8793-01737915a754 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.898707 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0a8e-9432-4043-8793-01737915a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.898985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0a8e-9432-4043-8793-01737915a754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.899684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0a8e-9432-4043-8793-01737915a754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.901332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0a8e-9432-4043-8793-01737915a754 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.915634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1239/2472] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:45.917356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:45.928722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6aaea-b901-456f-9af6-a627dabb9795 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:45.933634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:45.934297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:45.934643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:45.934862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] PATH_INFO: `/v3/projects/37d99d3c3027483398b352198ecf68c6/users/54868cb051f844ad9b2d02ed0075e296/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:45.936773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d99d3c3027483398b352198ecf68c6, user_id=54868cb051f844ad9b2d02ed0075e296, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:45.940004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:45.951028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1234/2473] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:45 2022] PUT /identity/v3/projects/37d99d3c3027483398b352198ecf68c6/users/54868cb051f844ad9b2d02ed0075e296/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:35:46.020388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:46.020602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:46.020875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:46.052504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] MFA Rules not processed for user `54868cb051f844ad9b2d02ed0075e296`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:46.053753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:46.131855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1240/2474] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:46.143794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:46.145694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:46.214354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee3e16-d8dd-4840-9ee5-79178b3391d0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:46.225332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf9955-8d52-4074-b1dd-2375c8ab21fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:46.225964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9955-8d52-4074-b1dd-2375c8ab21fd service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:46.226246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9955-8d52-4074-b1dd-2375c8ab21fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:46.226519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9955-8d52-4074-b1dd-2375c8ab21fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:46.236935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf9955-8d52-4074-b1dd-2375c8ab21fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:46.238586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf9955-8d52-4074-b1dd-2375c8ab21fd service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:46.249626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1235/2475] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:47.047749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:47.116187 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b3653-dd5f-4f96-ae3b-48dfd17ed98e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:47.120647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:47.121283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:47.121552 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:47.121828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] PATH_INFO: `/v3/users/bc5867aa24b84c469d33dbc609e3779e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:47.122960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5867aa24b84c469d33dbc609e3779e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:47.124900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:47.307990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5867aa24b84c469d33dbc609e3779e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:47.316250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5867aa24b84c469d33dbc609e3779e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:47.316890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5867aa24b84c469d33dbc609e3779e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:47.317516 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5867aa24b84c469d33dbc609e3779e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:47.323801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:47.325945 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5867aa24b84c469d33dbc609e3779e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:47.349695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1241/2476] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:47 2022] DELETE /identity/v3/users/bc5867aa24b84c469d33dbc609e3779e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:47.354693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:47.355499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:47.355499 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:47.377630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:47.379048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:47.448287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1236/2477] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:47.457749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:47.459163 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:47.536521 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2111e-f936-4f39-898f-ba152b5a2b85 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:47.540494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:47.541109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:47.541380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:47.541646 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:47.568503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:47.568503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:47.578330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1242/2478] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:47.912800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:47.914226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:47.933971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328d51-57ba-4fb2-b522-4e916c28b0cd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:47.938642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:47.940376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:47.940659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:47.941487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] PATH_INFO: `/v3/projects/bc0fc6eafc344b58b7fdb9cdcdc3a2e3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:47.942846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0fc6eafc344b58b7fdb9cdcdc3a2e3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:47.944790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:47.996988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0fc6eafc344b58b7fdb9cdcdc3a2e3', 'request_id': 'req-420819f1-53b3-4d9f-9526-7a008857c2ad'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.004738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0fc6eafc344b58b7fdb9cdcdc3a2e3', 'request_id': 'req-420819f1-53b3-4d9f-9526-7a008857c2ad'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.010639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0fc6eafc344b58b7fdb9cdcdc3a2e3', 'request_id': 'req-420819f1-53b3-4d9f-9526-7a008857c2ad'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.022822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] The token cache is being invalidate because project bc0fc6eafc344b58b7fdb9cdcdc3a2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:48.023115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.023825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.025331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1237/2479] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:47 2022] DELETE /identity/v3/projects/bc0fc6eafc344b58b7fdb9cdcdc3a2e3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:48.029367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:48.040212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f7604-f377-4e7f-a06f-d82e50503733 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:48.102098 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:48.102762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:48.102944 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:48.103208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] PATH_INFO: `/v3/users/54868cb051f844ad9b2d02ed0075e296` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:48.104364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54868cb051f844ad9b2d02ed0075e296)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:48.106254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:48.161702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54868cb051f844ad9b2d02ed0075e296'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.170885 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54868cb051f844ad9b2d02ed0075e296'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.171505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54868cb051f844ad9b2d02ed0075e296'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.172147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54868cb051f844ad9b2d02ed0075e296'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.180982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b559cd23-e1d9-4aec-89ed-8dc131ffc2d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54868cb051f844ad9b2d02ed0075e296'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.202478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:48.208695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1243/2480] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:35:48 2022] DELETE /identity/v3/users/54868cb051f844ad9b2d02ed0075e296 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:48.554380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:48.563995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:48.612079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420819f1-53b3-4d9f-9526-7a008857c2ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:48.621595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:48.734615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:48.735844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:48.737166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:48.737736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] PATH_INFO: `/v3/projects/37d99d3c3027483398b352198ecf68c6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:48.739399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d99d3c3027483398b352198ecf68c6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:48.741996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:48.792379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d99d3c3027483398b352198ecf68c6', 'request_id': 'req-0888f7db-4ca2-498c-ab59-7d43ceca9f24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.802974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d99d3c3027483398b352198ecf68c6', 'request_id': 'req-0888f7db-4ca2-498c-ab59-7d43ceca9f24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.811337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d99d3c3027483398b352198ecf68c6', 'request_id': 'req-0888f7db-4ca2-498c-ab59-7d43ceca9f24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.825105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] The token cache is being invalidate because project 37d99d3c3027483398b352198ecf68c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:35:48.825663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.826496 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0888f7db-4ca2-498c-ab59-7d43ceca9f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:35:48.828678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1238/2481] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:35:48 2022] DELETE /identity/v3/projects/37d99d3c3027483398b352198ecf68c6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:48.833569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:48.833849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:48.834107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:48.854077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:48.855285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:48.929911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1244/2482] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:48.937418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:48.937788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:48.938136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:48.961123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:48.962810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:48.975595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1239/2483] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:48.977041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:48.987133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a922a3b-aa6f-423b-af9a-4fe9a6c07a28 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:48.991836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:48.992461 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:48.992737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:48.993012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:48.993733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:35:48.994045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:48.995670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.009008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1245/2484] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:35:49.014304 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:49.025183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375d88a-5541-4582-a556-5803fbcd0571 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:49.030021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:49.030769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.031055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.031327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.032014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:49.034279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.067581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1240/2485] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:35:49 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:49.068633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:49.079127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2dd70-8f52-4f3f-a442-6294f14c7839 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:49.140502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:49.141309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.141418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.141641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.142359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:49.144309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.165794 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1246/2486] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:35:49 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:35:49.170127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:49.183305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8faf-292d-484f-8131-09f7243b3bb5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:49.187866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:49.188486 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.188757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.189047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.189866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:49.191478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.202973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1241/2487] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:49.207214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:49.218936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb77e7f-fb06-4de5-9dd8-109fb0be6328 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:49.224981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725ff8b-9ee9-4bf2-9720-34e5850fb793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:49.225602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725ff8b-9ee9-4bf2-9720-34e5850fb793 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.225881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725ff8b-9ee9-4bf2-9720-34e5850fb793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.226153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725ff8b-9ee9-4bf2-9720-34e5850fb793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.226799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725ff8b-9ee9-4bf2-9720-34e5850fb793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:49.228698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725ff8b-9ee9-4bf2-9720-34e5850fb793 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.240273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1247/2488] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:35:49.244236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:49.254894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a684468-3529-4d85-8378-8c9089a288d6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:49.259586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:49.260252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.260534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.260848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] PATH_INFO: `/v3/projects/f5742066e1ec40a9977ff47af0e95b3e/users/0dcbe926403c46979897ca21d42d0533/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.281819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5742066e1ec40a9977ff47af0e95b3e, user_id=0dcbe926403c46979897ca21d42d0533, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:49.285400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.332831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1242/2489] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:35:49 2022] PUT /identity/v3/projects/f5742066e1ec40a9977ff47af0e95b3e/users/0dcbe926403c46979897ca21d42d0533/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:35:49.345990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbff55-519c-4dc3-ad47-0169bec1383a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.346370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbff55-519c-4dc3-ad47-0169bec1383a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.346993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbff55-519c-4dc3-ad47-0169bec1383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.376487 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7bdbff55-519c-4dc3-ad47-0169bec1383a None None] MFA Rules not processed for user `0dcbe926403c46979897ca21d42d0533`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:35:49.377732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bdbff55-519c-4dc3-ad47-0169bec1383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:49.438127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:49.448508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1248/2490] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:35:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:35:49.460979 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:35:49.462790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:35:49.503791 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:49.530603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceeeff-5646-4fd2-a7a2-da8ae283b98b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:35:49.534535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:35:49.535147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:35:49.535425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:35:49.535725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:35:49.544291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:35:49.545730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:35:49.555805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1243/2491] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:35:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:35:59.446462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:35:59.551659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=11 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:01.214996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:01.215489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:01.215627 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:01.235991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:01.237242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:01.300366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1249/2492] 10.222.0.25 () {58 vars in 967 bytes} [Fri Sep 9 06:36:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:01.341946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:01.343759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:01.441967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b312d-2376-4394-829b-e789d478622e service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:01.449097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8da03a-125a-4b34-aa9f-2d4749a16688 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:01.449858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8da03a-125a-4b34-aa9f-2d4749a16688 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:01.450089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8da03a-125a-4b34-aa9f-2d4749a16688 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:01.450362 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8da03a-125a-4b34-aa9f-2d4749a16688 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:01.460081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8da03a-125a-4b34-aa9f-2d4749a16688 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:01.461264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8da03a-125a-4b34-aa9f-2d4749a16688 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:01.470336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1244/2493] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:36:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:36:08.411392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:08.475656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01b7c2-790a-48d3-bce4-84b2343bf3b1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:08.480113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:08.480747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:08.481022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:08.481291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] PATH_INFO: `/v3/users/0dcbe926403c46979897ca21d42d0533` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:08.482454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcbe926403c46979897ca21d42d0533)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:08.484371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:08.530337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcbe926403c46979897ca21d42d0533'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:08.538976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcbe926403c46979897ca21d42d0533'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:08.539597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcbe926403c46979897ca21d42d0533'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:08.540221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcbe926403c46979897ca21d42d0533'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:08.547326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcbe926403c46979897ca21d42d0533'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:08.565702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1250/2494] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:08 2022] DELETE /identity/v3/users/0dcbe926403c46979897ca21d42d0533 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:08.570329 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:08.570788 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:08.570886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:08.589848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:08.590925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:08.649214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1245/2495] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:08.659488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:08.660946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:08.688951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:08.718886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e698489-a43f-4735-9b04-fc75d25d6906 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:08.723390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd225675-07bf-44f8-b7c1-7df9da4f5352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:08.724007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd225675-07bf-44f8-b7c1-7df9da4f5352 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:08.724295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd225675-07bf-44f8-b7c1-7df9da4f5352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:08.724560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd225675-07bf-44f8-b7c1-7df9da4f5352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:08.733389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd225675-07bf-44f8-b7c1-7df9da4f5352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:08.734792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd225675-07bf-44f8-b7c1-7df9da4f5352 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:08.745056 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1251/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:09.017303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.018713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:09.031276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87f846-3270-4a22-8cf5-28f6cb68bf25 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.037947 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.038911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.039105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.039448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] PATH_INFO: `/v3/projects/f5742066e1ec40a9977ff47af0e95b3e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.040625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5742066e1ec40a9977ff47af0e95b3e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.042737 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.091116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5742066e1ec40a9977ff47af0e95b3e', 'request_id': 'req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:09.100209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5742066e1ec40a9977ff47af0e95b3e', 'request_id': 'req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:09.106170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5742066e1ec40a9977ff47af0e95b3e', 'request_id': 'req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:09.117760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] The token cache is being invalidate because project f5742066e1ec40a9977ff47af0e95b3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:09.118048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:09.118844 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3726a1fb-c0d4-43f0-b5e8-90c0ec54dcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:09.120443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1246/2497] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:09 2022] DELETE /identity/v3/projects/f5742066e1ec40a9977ff47af0e95b3e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:09.125118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.125467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.125845 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.145619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:09.146811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:09.205527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1252/2498] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:09.213975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.214268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.214545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.234380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:09.235611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:09.243945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1247/2499] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:09.248080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.258444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0afffb-43a7-471b-9673-c494bbcac5e6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.263129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.263757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.264040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.264308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.264981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:09.265282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.267460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.278935 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1253/2500] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:09.283010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.292650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553ed75-84ca-4cb3-a539-ffc78fdffec3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.297156 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.297783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.298062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.298332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.299072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.301006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.305363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:09.331012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1248/2501] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:09 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:09.335048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.345068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74203747-d167-4c45-b8c1-da1ee4337193 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.400806 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.401411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.401691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.401963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.402596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.404551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.425339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1254/2502] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:09 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:09.429242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.439996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902265-4216-4409-b2b3-b5e0f9deb0e0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.444602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.445202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.445481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.445754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.446398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.448073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.458578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1249/2503] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:09.462437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.472654 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40176c31-62de-45b5-9337-1c76dbf0c89b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.477226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0f386-e228-4a63-905a-b7a42f20d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.477830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f386-e228-4a63-905a-b7a42f20d59a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.478092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f386-e228-4a63-905a-b7a42f20d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.478365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0f386-e228-4a63-905a-b7a42f20d59a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.479002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f386-e228-4a63-905a-b7a42f20d59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.480637 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0f386-e228-4a63-905a-b7a42f20d59a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.491022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1255/2504] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:09.495093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.505886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c9e0f-6813-43eb-aef7-3baf4410a7be admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.510534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.511189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.511475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.511796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] PATH_INFO: `/v3/projects/88cc942a4f934321ba3d55d7a8c1da71/users/e4f62b25f8b84ee0a3acd7865525b5c1/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.527179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88cc942a4f934321ba3d55d7a8c1da71, user_id=e4f62b25f8b84ee0a3acd7865525b5c1, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.529857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.543037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1250/2505] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:09 2022] PUT /identity/v3/projects/88cc942a4f934321ba3d55d7a8c1da71/users/e4f62b25f8b84ee0a3acd7865525b5c1/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:09.553798 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.554085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.554370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.582564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] MFA Rules not processed for user `e4f62b25f8b84ee0a3acd7865525b5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:09.583837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:09.604192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:09.650237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1256/2506] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:09.661020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:09.662849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:09.720204 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a50a8-fc96-4263-947f-81862cfd84a7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:09.724309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3522d12-9025-4e75-9d1e-001f3cdc7b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:09.724909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522d12-9025-4e75-9d1e-001f3cdc7b70 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:09.725193 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522d12-9025-4e75-9d1e-001f3cdc7b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:09.725460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3522d12-9025-4e75-9d1e-001f3cdc7b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:09.733665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522d12-9025-4e75-9d1e-001f3cdc7b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:09.735611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3522d12-9025-4e75-9d1e-001f3cdc7b70 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:09.744370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1251/2507] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:19.647419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=17 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:19.740385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:25.982435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:26.050936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6998e6-baa5-4e3b-810a-bf3ea5f0d0b0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:26.055694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:26.056346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:26.056632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:26.056910 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] PATH_INFO: `/v3/users/e4f62b25f8b84ee0a3acd7865525b5c1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:26.058072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f62b25f8b84ee0a3acd7865525b5c1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:26.060069 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:26.112555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f62b25f8b84ee0a3acd7865525b5c1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.121481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f62b25f8b84ee0a3acd7865525b5c1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.122107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f62b25f8b84ee0a3acd7865525b5c1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.122689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f62b25f8b84ee0a3acd7865525b5c1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.131306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f62b25f8b84ee0a3acd7865525b5c1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.149341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1257/2508] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:25 2022] DELETE /identity/v3/users/e4f62b25f8b84ee0a3acd7865525b5c1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:26.154016 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:26.154556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:26.154556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:26.175854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:26.177222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:26.244740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1252/2509] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:26.254799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:26.256505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:26.316708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb32b3-46ea-4e84-9a05-51aae8542545 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:26.320494 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:26.321239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:26.321530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:26.321788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:26.332670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:26.334142 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:26.344663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1258/2510] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:26.657326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:26.658601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:26.671789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc0152-bb60-46ba-89d3-c9e7a2af73c2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:26.678192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:26.678912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:26.679420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:26.679778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] PATH_INFO: `/v3/projects/88cc942a4f934321ba3d55d7a8c1da71` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:26.681500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88cc942a4f934321ba3d55d7a8c1da71)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:26.683681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:26.754291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88cc942a4f934321ba3d55d7a8c1da71', 'request_id': 'req-47ce3359-1635-454a-a69a-f3913d4eae05'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.764455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88cc942a4f934321ba3d55d7a8c1da71', 'request_id': 'req-47ce3359-1635-454a-a69a-f3913d4eae05'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.800031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88cc942a4f934321ba3d55d7a8c1da71', 'request_id': 'req-47ce3359-1635-454a-a69a-f3913d4eae05'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.815385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] The token cache is being invalidate because project 88cc942a4f934321ba3d55d7a8c1da71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:26.815993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.816942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:26.819418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1253/2511] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:26 2022] DELETE /identity/v3/projects/88cc942a4f934321ba3d55d7a8c1da71 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:32.879093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:32.880622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:32.892576 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6a3a3-565b-4fb5-acc4-cf813152101b service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:32.912955 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:32.948868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:32.949473 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:32.949754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:32.950022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:32.951771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:33.016001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:33.017520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:33.027321 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1259/2512] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:34.486242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:34.487934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:34.546776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce3359-1635-454a-a69a-f3913d4eae05 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:34.550500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:34.551110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:34.551391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:34.551717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:34.553617 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:34.597248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:34.620197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:34.621805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:34.635538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1254/2513] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:36.273058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:36.274642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:36.293032 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f2c88-aa17-4e2f-8ebf-dbfb9007237f service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:36.298366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:36.299009 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.299284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.299574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] PATH_INFO: `/v3/users/925d5f23bd3f4b35a66736d2a6a3c3eb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.300756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=925d5f23bd3f4b35a66736d2a6a3c3eb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:36.302685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:36.353851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925d5f23bd3f4b35a66736d2a6a3c3eb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.362019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925d5f23bd3f4b35a66736d2a6a3c3eb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.362778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925d5f23bd3f4b35a66736d2a6a3c3eb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.363393 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925d5f23bd3f4b35a66736d2a6a3c3eb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.370483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04494008-55c4-456a-b35e-4f14e6163a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925d5f23bd3f4b35a66736d2a6a3c3eb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.390277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1260/2514] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:36 2022] DELETE /identity/v3/users/925d5f23bd3f4b35a66736d2a6a3c3eb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:36.562451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:36.580035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b0d26-028c-43e3-a54d-33b6f00f2a7f service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:36.637078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:36.637618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.637894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.638161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] PATH_INFO: `/v3/projects/e8ee1a9407994bf9a624e6231c7fe565` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.639260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8ee1a9407994bf9a624e6231c7fe565)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:36.641716 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:36.686877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ee1a9407994bf9a624e6231c7fe565', 'request_id': 'req-39884dc4-7771-412f-8da5-0f9ee8add006'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.695224 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ee1a9407994bf9a624e6231c7fe565', 'request_id': 'req-39884dc4-7771-412f-8da5-0f9ee8add006'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.700851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ee1a9407994bf9a624e6231c7fe565', 'request_id': 'req-39884dc4-7771-412f-8da5-0f9ee8add006'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.712528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] The token cache is being invalidate because project e8ee1a9407994bf9a624e6231c7fe565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:36.712813 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.713438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-39884dc4-7771-412f-8da5-0f9ee8add006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:36.714886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1255/2515] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:36 2022] DELETE /identity/v3/projects/e8ee1a9407994bf9a624e6231c7fe565 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:36.720825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.721110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.721378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.741399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:36.742584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:36.781949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:36.801992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1261/2516] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:36.809485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.809776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.810056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.830043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:36.831188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:36.839022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1256/2517] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:36.843208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:36.853726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c8a8c-6735-4ca7-97c3-40644a5608a1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:36.858248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:36.858841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.859119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.859407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.860071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:36.860379 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:36.861956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:36.873451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1262/2518] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:36:36.877790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:36.887361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bc267-55b3-449b-b8e2-03ea3a03ea86 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:36.892866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:36.893477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.893758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.894036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.894683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:36.896671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:36.927648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1257/2519] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:36 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:36.928748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:36.939439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd7ca4-2606-4085-a73e-1fa11287f5e1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:36.996799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:36.997405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:36.997683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:36.997954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:36.998588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.001161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.022543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1263/2520] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:36 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:37.026719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:37.037879 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbc18-7a47-460c-9d86-954a43d2a9e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:37.042665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:37.043264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.043567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.043843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.044633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.046596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.058149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1258/2521] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:37.061933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:37.072364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5bd43-70b6-4935-b503-bbe61a3099f8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:37.076879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2ea4-1053-4a65-8e19-e804a509b556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:37.077462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2ea4-1053-4a65-8e19-e804a509b556 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.078442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2ea4-1053-4a65-8e19-e804a509b556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.078706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2ea4-1053-4a65-8e19-e804a509b556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.079564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e2ea4-1053-4a65-8e19-e804a509b556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.081239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e2ea4-1053-4a65-8e19-e804a509b556 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.091613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1264/2522] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:37.095794 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:37.106063 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6cd9d-1043-4d8a-8d49-ed8d59ed25ed admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:37.110562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:37.111206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.111480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.111829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] PATH_INFO: `/v3/projects/26a6d8329ea74a779407f080c1fc8338/users/f6050f459a184bc0ae9739cb052b729d/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.126782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a6d8329ea74a779407f080c1fc8338, user_id=f6050f459a184bc0ae9739cb052b729d, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.129533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.142722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1259/2523] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:37 2022] PUT /identity/v3/projects/26a6d8329ea74a779407f080c1fc8338/users/f6050f459a184bc0ae9739cb052b729d/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:37.153025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.153312 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.153586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.182015 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] MFA Rules not processed for user `f6050f459a184bc0ae9739cb052b729d`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:37.183213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:37.249282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1265/2524] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:37.261007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:37.262724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:37.270109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:37.338601 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1175f-5fdf-4d00-afe5-cb32be447158 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:37.344133 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca49d8-b433-41ec-a58b-f88c29fd37bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:37.344728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca49d8-b433-41ec-a58b-f88c29fd37bb service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.345003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca49d8-b433-41ec-a58b-f88c29fd37bb service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.345270 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca49d8-b433-41ec-a58b-f88c29fd37bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.353197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca49d8-b433-41ec-a58b-f88c29fd37bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.354749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca49d8-b433-41ec-a58b-f88c29fd37bb service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.363845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1260/2525] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:37.590685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:37.635071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:37.658347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ffec-b614-4bb3-89a7-d38cccaaedf7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:37.662871 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:37.663502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.663829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.664101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] PATH_INFO: `/v3/users/f6050f459a184bc0ae9739cb052b729d` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.665221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6050f459a184bc0ae9739cb052b729d)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.667191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.716066 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6050f459a184bc0ae9739cb052b729d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:37.725245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6050f459a184bc0ae9739cb052b729d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:37.725840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6050f459a184bc0ae9739cb052b729d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:37.726770 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6050f459a184bc0ae9739cb052b729d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:37.735607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6050f459a184bc0ae9739cb052b729d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:37.753235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1266/2526] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:37 2022] DELETE /identity/v3/users/f6050f459a184bc0ae9739cb052b729d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:37.757719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.758186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.758285 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.778540 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:37.779759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:37.838295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1261/2527] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:37.849384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:37.850779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:37.916208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb0d4-38b2-4295-9def-4f8c6b4d7a6d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:37.919912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:37.920517 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:37.920797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:37.921070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:37.932787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:37.934428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:37.944649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1267/2528] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:38.247388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.248880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:38.268549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8daa0-a684-4ffa-8254-152fe52f5885 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.274840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.275652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.276159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.276644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] PATH_INFO: `/v3/projects/26a6d8329ea74a779407f080c1fc8338` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.278137 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a6d8329ea74a779407f080c1fc8338)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.280369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.377665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.388218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a6d8329ea74a779407f080c1fc8338', 'request_id': 'req-4902e8b3-f69c-448d-a48c-0545ac827e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:38.393841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f005e-6b6f-455c-8c53-f41ded86fe2f service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.397653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a6d8329ea74a779407f080c1fc8338', 'request_id': 'req-4902e8b3-f69c-448d-a48c-0545ac827e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:38.404888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a6d8329ea74a779407f080c1fc8338', 'request_id': 'req-4902e8b3-f69c-448d-a48c-0545ac827e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:38.423888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] The token cache is being invalidate because project 26a6d8329ea74a779407f080c1fc8338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:38.424292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:38.426886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4902e8b3-f69c-448d-a48c-0545ac827e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:38.430990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1262/2529] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:38 2022] DELETE /identity/v3/projects/26a6d8329ea74a779407f080c1fc8338 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:38.436179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3b95-dc8f-44e4-897a-379c62fce99f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.436802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3b95-dc8f-44e4-897a-379c62fce99f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.436931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3b95-dc8f-44e4-897a-379c62fce99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.469611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-482d3b95-dc8f-44e4-897a-379c62fce99f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:38.470846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-482d3b95-dc8f-44e4-897a-379c62fce99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:38.487099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:38.499956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.500949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.501230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.501500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.503539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:38.527982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:38.568434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1263/2530] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:38.576227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.576588 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.576869 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.595815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.597274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:38.597507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.598474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:38.606507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1264/2531] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:38.607698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1268/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:36:38.611369 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.620515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322aaad-e6fb-4410-b1ed-0b09bc77c73c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.625569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.626174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.626455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.626727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.627383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:38.627729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.629333 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.641145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1265/2533] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:38.645555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.655366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5c7fa-4777-440b-bf6f-7d31a16a7781 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.661094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.661708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.661983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.662272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.662889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.664827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.697399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1269/2534] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:38 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:38.698567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.709518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d54da3-dfac-4cf6-90b0-b3e07864d688 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.771664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.772341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.772545 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.772815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.773451 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.775381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.802003 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1266/2535] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:38 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:38.803237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.814727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b581934-197e-4149-ae08-19831e1e5f08 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.819353 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.819995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.820270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.820536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.821347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.822932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.835245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1270/2536] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:38.839594 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.850927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7b4d-018f-4ded-8eb5-d9e03b0e89af admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.855694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169ec6c-b617-47dd-b627-099963e65251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.856335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169ec6c-b617-47dd-b627-099963e65251 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.856614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169ec6c-b617-47dd-b627-099963e65251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.856913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169ec6c-b617-47dd-b627-099963e65251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.857631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169ec6c-b617-47dd-b627-099963e65251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.859698 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169ec6c-b617-47dd-b627-099963e65251 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.871251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1267/2537] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:38.876107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:38.886218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c6b8-dea9-476a-bf71-935ee17985c5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:38.891908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:38.892596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.892870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.893138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] PATH_INFO: `/v3/projects/63e89bfd18cf48c7af8acd21d567e499/users/5fc1e72011e848faa92c5ce1f51ea533/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.908555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e89bfd18cf48c7af8acd21d567e499, user_id=5fc1e72011e848faa92c5ce1f51ea533, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:38.911172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:38.924380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1271/2538] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:38 2022] PUT /identity/v3/projects/63e89bfd18cf48c7af8acd21d567e499/users/5fc1e72011e848faa92c5ce1f51ea533/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:38.935081 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:38.935423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:38.935725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:38.964182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] MFA Rules not processed for user `5fc1e72011e848faa92c5ce1f51ea533`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:38.965389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:39.034581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1268/2539] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:39.039185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.105118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eae041-4017-4825-a567-ca28bf2013b8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.110435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.111003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.111284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.111568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.112204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.114121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.142282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1272/2540] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:39 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:39.146724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.152448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:39.160060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b894c-1e88-4abb-81eb-a0a172584f75 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.217127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.217742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.218023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.218336 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.219190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.221147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.247821 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1269/2541] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:39 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:39.249070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.259973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abab84-8ba4-4283-a5e6-732816fbd337 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.264427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.265016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.266004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.266284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.266913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.268524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.279152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1273/2542] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:39.283361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.293632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d126b-b331-4527-a147-341d95ed0e23 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.298382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.299046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.299324 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.299632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] PATH_INFO: `/v3/projects/6a380fd7d49147789dc0919d76585cf7/users/916de33cdde34fdcb39253b5744e687a/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.317587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a380fd7d49147789dc0919d76585cf7, user_id=916de33cdde34fdcb39253b5744e687a, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.320326 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.334032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1270/2543] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:39 2022] PUT /identity/v3/projects/6a380fd7d49147789dc0919d76585cf7/users/916de33cdde34fdcb39253b5744e687a/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:39.338583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.349719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a0f0a-1eb3-4960-94e2-4972337a2bf0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.376950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:39.410055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef4b7d-8513-4980-a593-ff26d39c71b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.410659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef4b7d-8513-4980-a593-ff26d39c71b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.410933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef4b7d-8513-4980-a593-ff26d39c71b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.411200 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef4b7d-8513-4980-a593-ff26d39c71b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.411870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef4b7d-8513-4980-a593-ff26d39c71b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.413482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef4b7d-8513-4980-a593-ff26d39c71b2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.424320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1274/2544] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:39.428524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.439161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39a755-514c-443c-87e5-1fc301b8cc7e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.443717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.444370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.444651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.444922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] PATH_INFO: `/v3/projects/6a380fd7d49147789dc0919d76585cf7/users/916de33cdde34fdcb39253b5744e687a/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.446847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a380fd7d49147789dc0919d76585cf7, user_id=916de33cdde34fdcb39253b5744e687a, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.449535 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.460655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1271/2545] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:39 2022] PUT /identity/v3/projects/6a380fd7d49147789dc0919d76585cf7/users/916de33cdde34fdcb39253b5744e687a/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:39.473339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.473721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.474072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.506253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] MFA Rules not processed for user `916de33cdde34fdcb39253b5744e687a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:39.507582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:39.591395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1275/2546] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:39.605308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:39.607185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:39.683639 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b29ad9-23f0-4761-a315-211068b603e3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:39.687255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:39.687985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:39.688280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:39.688552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:39.706028 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:39.707334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:39.723599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1272/2547] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:40.086281 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:40.095976 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870769ee-2acb-4fb7-8bd2-c2df22ebb39f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:40.100870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '916de33cdde34fdcb39253b5744e687a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a380fd7d49147789dc0919d76585cf7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:40.101475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:40.101763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:40.102040 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:40.102690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:40.102994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:40.104616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:40.116589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1276/2548] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:36:40.120688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:40.130602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba378c-2e83-4854-92ea-f73b0fc95fe1 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:40.135344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '916de33cdde34fdcb39253b5744e687a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a380fd7d49147789dc0919d76585cf7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:40.136065 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:40.136415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:40.136756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:40.137474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:40.139461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:40.176765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1273/2549] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:40 2022] POST /identity/v3/projects => generated 354 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:40.180823 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:40.191394 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fce2a2-00f3-468f-ad36-8f0903c72425 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:40.263364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '916de33cdde34fdcb39253b5744e687a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a380fd7d49147789dc0919d76585cf7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:40.264184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:40.264796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:40.266209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:40.267268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:40.267783 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:40.269588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2336287-7c1f-4722-ad14-f764969360ba tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:40.288186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1277/2550] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:40.344384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:40.356621 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632af256-655b-444d-9dbc-68e97695bbb2 tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:40.369649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '916de33cdde34fdcb39253b5744e687a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a380fd7d49147789dc0919d76585cf7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:40.370431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:40.370717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:40.371691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] PATH_INFO: `/v3/projects/d16472e0220f4ed09d7922795e6da020` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:40.372945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d16472e0220f4ed09d7922795e6da020)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:40.376295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:40.427498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:40.444491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16472e0220f4ed09d7922795e6da020', 'request_id': 'req-903beecc-6f92-4b5d-94b2-2f26cb7e503e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:40.455210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16472e0220f4ed09d7922795e6da020', 'request_id': 'req-903beecc-6f92-4b5d-94b2-2f26cb7e503e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:40.463726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16472e0220f4ed09d7922795e6da020', 'request_id': 'req-903beecc-6f92-4b5d-94b2-2f26cb7e503e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:40.482877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] The token cache is being invalidate because project d16472e0220f4ed09d7922795e6da020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:40.483315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:40.484218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:40.487082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1274/2551] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:40 2022] DELETE /identity/v3/projects/d16472e0220f4ed09d7922795e6da020 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:40.702344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ddcaa-5764-45d2-b987-feebace7cdac None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:40.702837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ddcaa-5764-45d2-b987-feebace7cdac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:40.703377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ddcaa-5764-45d2-b987-feebace7cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:40.729471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0d2ddcaa-5764-45d2-b987-feebace7cdac None None] MFA Rules not processed for user `472b6623f57a41568fe7ffce8e40e350`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:40.730931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0d2ddcaa-5764-45d2-b987-feebace7cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:40.765838 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:40.814053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:40.818291 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1278/2552] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:40.818446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:40.932230 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903beecc-6f92-4b5d-94b2-2f26cb7e503e tempest-VolumeQuotaClassesTest-590733520 tempest-VolumeQuotaClassesTest-590733520-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:40.938145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:40.939128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:40.939478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:40.939842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] PATH_INFO: `/v3/users/5fc1e72011e848faa92c5ce1f51ea533` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:40.941382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fc1e72011e848faa92c5ce1f51ea533)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:40.946666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:41.019462 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fc1e72011e848faa92c5ce1f51ea533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:41.032453 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fc1e72011e848faa92c5ce1f51ea533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:41.033348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fc1e72011e848faa92c5ce1f51ea533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:41.043101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fc1e72011e848faa92c5ce1f51ea533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:41.058359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fc1e72011e848faa92c5ce1f51ea533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:41.066987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1275/2553] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:40 2022] DELETE /identity/v3/users/5fc1e72011e848faa92c5ce1f51ea533 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:41.071530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:41.071842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:41.072119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:41.094824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:41.096270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:41.170039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1279/2554] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:41.190635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:41.192134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:41.241392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:41.276339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b3f5b-e938-4fd2-bfd6-7f366ce59c32 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:41.282722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:41.283831 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:41.284236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:41.285070 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:41.297894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:41.301985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:41.316404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1276/2555] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:36:42.020518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:42.021810 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:42.036827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b758be-7650-47a8-a74c-ca89e5d4e55d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:42.041761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:42.042484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:42.042767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:42.043048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] PATH_INFO: `/v3/projects/63e89bfd18cf48c7af8acd21d567e499` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:42.048825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e89bfd18cf48c7af8acd21d567e499)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:42.051978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:42.134222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e89bfd18cf48c7af8acd21d567e499', 'request_id': 'req-104241bd-114f-4cf0-8e8b-c276858d44a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.147977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e89bfd18cf48c7af8acd21d567e499', 'request_id': 'req-104241bd-114f-4cf0-8e8b-c276858d44a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.165719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e89bfd18cf48c7af8acd21d567e499', 'request_id': 'req-104241bd-114f-4cf0-8e8b-c276858d44a6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.205612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] The token cache is being invalidate because project 63e89bfd18cf48c7af8acd21d567e499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:42.205920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.208095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.211210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1280/2556] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:42 2022] DELETE /identity/v3/projects/63e89bfd18cf48c7af8acd21d567e499 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:42.215645 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:42.251948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a305e-4e55-46ec-8814-f69f32b5a474 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:42.358489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:42.359515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:42.360378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:42.360817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] PATH_INFO: `/v3/users/916de33cdde34fdcb39253b5744e687a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:42.362169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916de33cdde34fdcb39253b5744e687a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:42.365238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:42.467470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916de33cdde34fdcb39253b5744e687a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.498644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916de33cdde34fdcb39253b5744e687a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.500302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916de33cdde34fdcb39253b5744e687a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.513171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916de33cdde34fdcb39253b5744e687a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.534265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916de33cdde34fdcb39253b5744e687a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:42.544739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1277/2557] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:42 2022] DELETE /identity/v3/users/916de33cdde34fdcb39253b5744e687a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:42.987968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:42.991900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.008222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104241bd-114f-4cf0-8e8b-c276858d44a6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.055565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.059041 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.128700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.129427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.129813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.130251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] PATH_INFO: `/v3/users/1c86ccfec70340c797ecba83163ff1a2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.131481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c86ccfec70340c797ecba83163ff1a2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.135990 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.138675 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:43.161790 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5ac51-7796-446b-a0f1-6a398aea8bd2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.169221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.170566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.170889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.171910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] PATH_INFO: `/v3/projects/6a380fd7d49147789dc0919d76585cf7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.173608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a380fd7d49147789dc0919d76585cf7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.175710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.178027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:43.209563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c86ccfec70340c797ecba83163ff1a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.217689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c86ccfec70340c797ecba83163ff1a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.218892 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c86ccfec70340c797ecba83163ff1a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.219520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c86ccfec70340c797ecba83163ff1a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.227320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-56b75cb9-0e9d-4841-9108-05b15b129a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c86ccfec70340c797ecba83163ff1a2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.235912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a380fd7d49147789dc0919d76585cf7', 'request_id': 'req-573d1df3-d3f0-4152-91ad-a4d383c4a61b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.245785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a380fd7d49147789dc0919d76585cf7', 'request_id': 'req-573d1df3-d3f0-4152-91ad-a4d383c4a61b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.249064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1281/2558] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:42 2022] DELETE /identity/v3/users/1c86ccfec70340c797ecba83163ff1a2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:43.253593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a380fd7d49147789dc0919d76585cf7', 'request_id': 'req-573d1df3-d3f0-4152-91ad-a4d383c4a61b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.267366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] The token cache is being invalidate because project 6a380fd7d49147789dc0919d76585cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:43.267695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.268371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-573d1df3-d3f0-4152-91ad-a4d383c4a61b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.269853 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1278/2559] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:43 2022] DELETE /identity/v3/projects/6a380fd7d49147789dc0919d76585cf7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:43.274723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.277197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.277481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.300988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:43.300988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.370361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1282/2560] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:43.378490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.378840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.379186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.400418 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:43.401742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.410905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1279/2561] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:43.415236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.425834 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe025bc-d33f-4618-83c4-573c8a933916 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.430788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.431396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.431713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.431989 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.432722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:43.433030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.434975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.436004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.437673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.449544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1283/2562] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:43.461395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.468983 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797fdd3-b7ee-4215-a366-ca24b29190c3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.474678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.475629 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.476150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.476583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] PATH_INFO: `/v3/projects/7a732015d4244c378e0204e1d216f57b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.478282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a732015d4244c378e0204e1d216f57b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.482135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.484514 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3f84d-d320-4db3-9c2e-26bda1acdaa3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.491217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.491800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.492085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.492771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.493412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.496802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.536344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1284/2563] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:43.540493 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.551710 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a732015d4244c378e0204e1d216f57b', 'request_id': 'req-e8f10ed4-0967-41e6-a974-3238211a0310'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.568321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a732015d4244c378e0204e1d216f57b', 'request_id': 'req-e8f10ed4-0967-41e6-a974-3238211a0310'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.575724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a732015d4244c378e0204e1d216f57b', 'request_id': 'req-e8f10ed4-0967-41e6-a974-3238211a0310'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.589429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] The token cache is being invalidate because project 7a732015d4244c378e0204e1d216f57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:43.589727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.590439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e8f10ed4-0967-41e6-a974-3238211a0310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:43.592347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1280/2564] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:43 2022] DELETE /identity/v3/projects/7a732015d4244c378e0204e1d216f57b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:43.597337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91570141-f81b-4e4d-bd66-4b735f170496 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.597738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91570141-f81b-4e4d-bd66-4b735f170496 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.598128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91570141-f81b-4e4d-bd66-4b735f170496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.615535 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:43.628141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-91570141-f81b-4e4d-bd66-4b735f170496 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:43.629551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91570141-f81b-4e4d-bd66-4b735f170496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.640118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a5b73-1eb3-4401-aafb-f1b1bb676a05 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.716269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1281/2565] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:43.725378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.725666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.725942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.733542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.734791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.735277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.735509 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.736214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.739388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.779881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:43.785489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1285/2566] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:43.789934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.792018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.801685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:43.804146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:43.813680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1282/2567] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:43.818711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.826381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18996806-47e5-4a3c-92de-dc16a1e17112 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.833706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20dbcc-adfb-4893-bed0-651b814519e7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.841208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.841854 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.842128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.842450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.844196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.845178 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.845824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.846084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.846685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.847022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.847802 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:43.848173 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.850012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.865771 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1283/2568] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:43.869912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.875934 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1286/2569] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:43.878332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.893447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1bc11-dde2-4681-ac9c-d8f8c1896321 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.894826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036d6b0-4f42-4acf-b2ef-179244947aa8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:43.899967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.900804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.901191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.902281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.902607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.905119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.906128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:43.907181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:43.908594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:43.909232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:43.910218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:43.914687 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:43.935867 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1287/2570] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:43.943582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1284/2571] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:43.943809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.944662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:43.959833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4291e13-33ab-49c4-abc2-33a876309f69 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.054571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead82a3b-632e-4bf3-b64d-ec749e0c5c5e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.061241 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.061500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8378bc7-4562-4be0-bc7e-75f7028d571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.062850 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.063184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.063415 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.063958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378bc7-4562-4be0-bc7e-75f7028d571c admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.064095 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.064660 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378bc7-4562-4be0-bc7e-75f7028d571c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.064948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8378bc7-4562-4be0-bc7e-75f7028d571c admin admin] PATH_INFO: `/v3/projects/1b320cda27624038afc642cc1f282931/users/b76ab85dc6804ddd882ef1b7d5b7dc67/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.066909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.090186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378bc7-4562-4be0-bc7e-75f7028d571c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b320cda27624038afc642cc1f282931, user_id=b76ab85dc6804ddd882ef1b7d5b7dc67, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.094832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1285/2572] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:43 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:44.095415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8378bc7-4562-4be0-bc7e-75f7028d571c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.099251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.112250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50626f90-dfb9-455d-ba98-a0f5a48f3eb5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.116549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1288/2573] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:43 2022] PUT /identity/v3/projects/1b320cda27624038afc642cc1f282931/users/b76ab85dc6804ddd882ef1b7d5b7dc67/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:44.126984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.127398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.127757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.165180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] MFA Rules not processed for user `b76ab85dc6804ddd882ef1b7d5b7dc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:44.167031 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:44.208532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:44.210181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.211258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.211635 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.212078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.213526 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.215383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.233523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1286/2574] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:44.237255 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.252039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49370f-56b3-4336-a548-6b2391c19b04 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.259464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1289/2575] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:44.261885 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.263973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.264652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.266075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.266795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.267812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.269854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.283694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1287/2576] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:44.288914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.289420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3ba21-042f-4c94-9ef0-a4e7011009c6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.294749 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.295283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.295570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.296017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.296750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.298861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.316752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7b549-8de6-43ed-bc8f-c1c044029100 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.322586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c784835-d249-48bf-bca5-bc808a816b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.323736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c784835-d249-48bf-bca5-bc808a816b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.324107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c784835-d249-48bf-bca5-bc808a816b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.324515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c784835-d249-48bf-bca5-bc808a816b99 admin admin] PATH_INFO: `/v3/projects/749eca06a6484ebba440959ae22a5858/users/8e26d2c775eb4b019be065da57ce4999/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.340148 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1290/2577] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:44 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:44.341724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.352267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c784835-d249-48bf-bca5-bc808a816b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749eca06a6484ebba440959ae22a5858, user_id=8e26d2c775eb4b019be065da57ce4999, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.355214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c784835-d249-48bf-bca5-bc808a816b99 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.375570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1288/2578] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:44 2022] PUT /identity/v3/projects/749eca06a6484ebba440959ae22a5858/users/8e26d2c775eb4b019be065da57ce4999/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:44.387158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.387507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.387748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.434303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] MFA Rules not processed for user `8e26d2c775eb4b019be065da57ce4999`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:44.436456 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:44.443251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:44.475950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293353-3624-4700-81a4-b91d0363c817 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.567782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1289/2579] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:44.576963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.578883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:44.648162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.648936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.649759 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.656231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.658728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.661356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.711309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1291/2580] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:44 2022] POST /identity/v3/users => generated 293 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:44.720396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.736005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd9c1d-d69a-4296-a1f9-b3dc1f7bd186 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.750744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36746559-f0b8-4392-8638-4328d8ea8947 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.758128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.758851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.759421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.760011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.761628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.762257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.762536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.762825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.763457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.767339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.773228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.775029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.782492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1292/2581] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:44.786688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.800892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1290/2582] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:44.812441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de628d-6c6e-43c8-b05e-4a2df28e1eb2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:44.818171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:44.819616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:44.819616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:44.819616 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] PATH_INFO: `/v3/projects/61adaa8a4979454480f37a6b2fcef7e3/users/1b39a969da864e8eb7c39b571b2b2a54/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:44.838859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61adaa8a4979454480f37a6b2fcef7e3, user_id=1b39a969da864e8eb7c39b571b2b2a54, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:44.841488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:44.861946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1293/2583] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:44 2022] PUT /identity/v3/projects/61adaa8a4979454480f37a6b2fcef7e3/users/1b39a969da864e8eb7c39b571b2b2a54/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:44.865686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:44.895510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d20d3-8985-48c6-9340-13f53973ddd5 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:45.010010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da2090-907f-41d5-80b4-2542e56392b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:45.012346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da2090-907f-41d5-80b4-2542e56392b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:45.012646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da2090-907f-41d5-80b4-2542e56392b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:45.012923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da2090-907f-41d5-80b4-2542e56392b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:45.014223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da2090-907f-41d5-80b4-2542e56392b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:45.017431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da2090-907f-41d5-80b4-2542e56392b7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:45.034815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1291/2584] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:45.039076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:45.054895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cf2e0-c3af-4416-afc9-9eb23bffdc66 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:45.064551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:45.065549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:45.066070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:45.066577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] PATH_INFO: `/v3/projects/61adaa8a4979454480f37a6b2fcef7e3/users/1b39a969da864e8eb7c39b571b2b2a54/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:45.070129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61adaa8a4979454480f37a6b2fcef7e3, user_id=1b39a969da864e8eb7c39b571b2b2a54, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:45.074705 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:45.083609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:45.093260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1294/2585] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:45 2022] PUT /identity/v3/projects/61adaa8a4979454480f37a6b2fcef7e3/users/1b39a969da864e8eb7c39b571b2b2a54/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:45.118142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:45.118664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:45.119249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:45.154906 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] MFA Rules not processed for user `1b39a969da864e8eb7c39b571b2b2a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:45.157393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:45.179489 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:45.261726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1292/2586] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:45.273047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:45.275986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:45.351666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97cec8-7394-48d0-99c8-8b2ff4fadd3b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:45.357821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:45.358441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:45.358744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:45.359034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:45.420193 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:45.422033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:45.431307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1295/2587] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:47.296580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:47.298011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:47.353408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb438e6-2fb4-447a-9121-b1dabf1f3ebe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:47.356929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:47.357527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:47.357808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:47.358079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:47.360348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:47.430494 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:47.431990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f42b16-2611-437f-a45e-0b21d6ff906f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:47.441898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1293/2588] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:50.131750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:50.144465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12acd46-1a16-4c5d-9e5d-87c48a6fecca service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:50.149205 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:50.149842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:50.150124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:50.150400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] PATH_INFO: `/v3/users/b76ab85dc6804ddd882ef1b7d5b7dc67` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:50.151827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76ab85dc6804ddd882ef1b7d5b7dc67)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:50.153802 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:50.205772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76ab85dc6804ddd882ef1b7d5b7dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:50.214036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76ab85dc6804ddd882ef1b7d5b7dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:50.214780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76ab85dc6804ddd882ef1b7d5b7dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:50.215425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76ab85dc6804ddd882ef1b7d5b7dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:50.223218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76ab85dc6804ddd882ef1b7d5b7dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:50.241921 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1296/2589] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:50 2022] DELETE /identity/v3/users/b76ab85dc6804ddd882ef1b7d5b7dc67 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:50.246491 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:50.246935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:50.247037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:50.278858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:50.280433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:50.346150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1294/2590] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:50.356045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:50.357541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:50.392234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:50.439888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88749da-1f3f-4c18-b7e2-46c890678da3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:50.444497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:50.445250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:50.445625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:50.445993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:50.456549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:50.458068 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:50.475495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1297/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:51.229559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:51.232221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.242600 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:51.254205 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46d395-09b3-4ab9-a8f1-836430449c2d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:51.267664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:51.268439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.268978 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.269355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] PATH_INFO: `/v3/users/0a6757e5408e40cc85ad6145064d8ba7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.270654 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6757e5408e40cc85ad6145064d8ba7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:51.273078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:51.273843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:51.277124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.324480 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f588be3-06b6-43a7-9e2b-da41f4534872 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:51.333244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:51.334004 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.334295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.334571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] PATH_INFO: `/v3/projects/1b320cda27624038afc642cc1f282931` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.335740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b320cda27624038afc642cc1f282931)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:51.337649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:51.340832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6757e5408e40cc85ad6145064d8ba7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.351934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6757e5408e40cc85ad6145064d8ba7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.352704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6757e5408e40cc85ad6145064d8ba7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.364289 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6757e5408e40cc85ad6145064d8ba7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.381276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6757e5408e40cc85ad6145064d8ba7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.387614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1295/2592] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:51 2022] DELETE /identity/v3/users/0a6757e5408e40cc85ad6145064d8ba7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:51.406308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b320cda27624038afc642cc1f282931', 'request_id': 'req-1e0487a1-92da-435b-a8e1-14cf0543d5d7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.413026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b320cda27624038afc642cc1f282931', 'request_id': 'req-1e0487a1-92da-435b-a8e1-14cf0543d5d7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.420577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b320cda27624038afc642cc1f282931', 'request_id': 'req-1e0487a1-92da-435b-a8e1-14cf0543d5d7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.434151 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] The token cache is being invalidate because project 1b320cda27624038afc642cc1f282931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:51.434366 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.436763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.438272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1298/2593] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:51 2022] DELETE /identity/v3/projects/1b320cda27624038afc642cc1f282931 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:51.442287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:51.443833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.520974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3cb27-753c-41f8-8b22-805cf84d0823 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:51.525927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:51.526683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.527251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.527662 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] PATH_INFO: `/v3/users/1b39a969da864e8eb7c39b571b2b2a54` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.528956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b39a969da864e8eb7c39b571b2b2a54)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:51.531651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:51.586292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b39a969da864e8eb7c39b571b2b2a54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.586477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:51.588036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.597653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b39a969da864e8eb7c39b571b2b2a54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.598343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b39a969da864e8eb7c39b571b2b2a54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.606043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b39a969da864e8eb7c39b571b2b2a54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.619168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:51.621926 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-84a857b8-308d-4477-931b-ed604b4ef3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b39a969da864e8eb7c39b571b2b2a54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.630612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1296/2594] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:51 2022] DELETE /identity/v3/users/1b39a969da864e8eb7c39b571b2b2a54 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:51.681946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0487a1-92da-435b-a8e1-14cf0543d5d7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:51.738893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:51.746197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:51.747308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.747766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.748110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] PATH_INFO: `/v3/projects/7d5de98a77f64cd5beeab6865f5ea7c2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.749581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5de98a77f64cd5beeab6865f5ea7c2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:51.751930 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:51.817086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5de98a77f64cd5beeab6865f5ea7c2', 'request_id': 'req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.822902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5de98a77f64cd5beeab6865f5ea7c2', 'request_id': 'req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.831844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5de98a77f64cd5beeab6865f5ea7c2', 'request_id': 'req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.844558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] The token cache is being invalidate because project 7d5de98a77f64cd5beeab6865f5ea7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:51.844933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.845607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:51.847477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1299/2595] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:51 2022] DELETE /identity/v3/projects/7d5de98a77f64cd5beeab6865f5ea7c2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:51.852394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583fc6-cd70-43d3-934f-3d2052310cbe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.852786 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583fc6-cd70-43d3-934f-3d2052310cbe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.853121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583fc6-cd70-43d3-934f-3d2052310cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.872916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c8583fc6-cd70-43d3-934f-3d2052310cbe None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:51.874299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c8583fc6-cd70-43d3-934f-3d2052310cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.938861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:51.940964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.943082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1297/2596] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:51.955936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.956422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.956874 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.957160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa000c-fa0f-48c7-a3bc-47734e99fc1c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:51.962132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:51.962825 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:51.963633 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:51.963913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] PATH_INFO: `/v3/projects/61adaa8a4979454480f37a6b2fcef7e3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:51.965054 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61adaa8a4979454480f37a6b2fcef7e3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:51.967237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:51.988346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:51.989677 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:51.998082 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1298/2597] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:52.003171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.012214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9203807-9aba-4376-a0de-652ec33ae909 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.019093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61adaa8a4979454480f37a6b2fcef7e3', 'request_id': 'req-c43b973e-9006-4abd-bd8a-6280bc1211e9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.024837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61adaa8a4979454480f37a6b2fcef7e3', 'request_id': 'req-c43b973e-9006-4abd-bd8a-6280bc1211e9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.032043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61adaa8a4979454480f37a6b2fcef7e3', 'request_id': 'req-c43b973e-9006-4abd-bd8a-6280bc1211e9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.044547 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] The token cache is being invalidate because project 61adaa8a4979454480f37a6b2fcef7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:52.044844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.045630 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c43b973e-9006-4abd-bd8a-6280bc1211e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.047836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1300/2598] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:51 2022] DELETE /identity/v3/projects/61adaa8a4979454480f37a6b2fcef7e3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:52.052740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc14de8-f514-4ec9-b596-fdd714a6d93f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.053061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc14de8-f514-4ec9-b596-fdd714a6d93f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.053370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc14de8-f514-4ec9-b596-fdd714a6d93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.073734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.074109 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bbc14de8-f514-4ec9-b596-fdd714a6d93f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:52.074346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.074625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.074921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.075335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bbc14de8-f514-4ec9-b596-fdd714a6d93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.075647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:52.076007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.078426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.084556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1301/2599] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:52.090616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1299/2600] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:52.092399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.093222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.093569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.094759 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.096787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.109103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e731d7a-a18f-4587-812f-e9817ae8fe3f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.113768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.113954 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:52.114439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.114719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.114989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.115095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.115642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.117516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.122868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1302/2601] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:52.127652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.136878 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc6a0-17ae-4ddd-b950-a86881efdae2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.142400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.142996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.143273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.143564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.144222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:52.144522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.148932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1300/2602] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:52.149081 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.150071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.161596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1303/2603] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:36:52.162482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.172166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d16c-8fe0-4f10-8c3d-64306b341eea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.212443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f83f33-9fde-493e-8868-00ce611d5583 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.215204 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:52.217258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.218018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.218298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.218715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.219367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.221284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.230146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:52.234199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.234997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.235270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.235568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.236269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.238199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.244841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1301/2604] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:52.248583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.258851 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbb3f-fa98-4fda-9e03-7f02ab4ec083 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.263798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.264398 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.265356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.265837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.266474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.268103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.271588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1304/2605] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:52.275856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.281833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1302/2606] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:52.285688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.296436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3733-8979-46e1-9e2d-795a5a896626 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.345772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa9248-c0ac-441c-a5e7-747d2832ddc7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.351119 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.352087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.352789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.353252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.354307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.356475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.367852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.368403 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.368691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.368963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.369623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.372086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.387412 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1303/2607] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:52.389158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1305/2608] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:52.393696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.393830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.405551 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645b920-5ffe-4ab9-9038-fb6464203f2e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.410952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687b44-a523-4632-bc1c-ff61b0ea891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.411740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687b44-a523-4632-bc1c-ff61b0ea891f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.412217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687b44-a523-4632-bc1c-ff61b0ea891f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.412554 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687b44-a523-4632-bc1c-ff61b0ea891f admin admin] PATH_INFO: `/v3/projects/399e4417c8ca47ccaa83f9debb714404/users/d9065f1aadab4e389bc91ab9f14762e0/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.417736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe947ba-a247-4743-a2d0-4fd352d5d264 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.429433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.430062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.430441 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.430807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.431563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.433342 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687b44-a523-4632-bc1c-ff61b0ea891f admin admin] RBAC: Authorizing `identity:create_grant(project_id=399e4417c8ca47ccaa83f9debb714404, user_id=d9065f1aadab4e389bc91ab9f14762e0, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.433610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.438315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687b44-a523-4632-bc1c-ff61b0ea891f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.449181 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1306/2609] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:52.453071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.462733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1304/2610] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:52 2022] PUT /identity/v3/projects/399e4417c8ca47ccaa83f9debb714404/users/d9065f1aadab4e389bc91ab9f14762e0/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:52.470723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af35538-5e2f-4c95-9776-fcd69052d299 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.474971 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.475298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.484787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.518721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] MFA Rules not processed for user `d9065f1aadab4e389bc91ab9f14762e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:52.520066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.548686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.549207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.549482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.549781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.550900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.552524 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.568858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1307/2611] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:52.568858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.568858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.581799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7160d6-621d-4f12-9a23-ded4a407becc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.586808 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.587439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.587751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.588020 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] PATH_INFO: `/v3/users/8e26d2c775eb4b019be065da57ce4999` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.595889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e26d2c775eb4b019be065da57ce4999)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.598046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.601660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1305/2612] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:52.604822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.623348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceecd86-3ab9-443f-bd55-334340525b61 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.630667 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.631354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.631663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.631938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] PATH_INFO: `/v3/projects/05b5c1aa83064941ba8a46fcea497d30/users/e11f10329ffc4d2dba60f865786d23ef/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.645481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:52.648268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b5c1aa83064941ba8a46fcea497d30, user_id=e11f10329ffc4d2dba60f865786d23ef, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.651442 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.653138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e26d2c775eb4b019be065da57ce4999'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.660902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e26d2c775eb4b019be065da57ce4999'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.661621 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e26d2c775eb4b019be065da57ce4999'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.662299 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e26d2c775eb4b019be065da57ce4999'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.665652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1306/2613] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:52 2022] PUT /identity/v3/projects/05b5c1aa83064941ba8a46fcea497d30/users/e11f10329ffc4d2dba60f865786d23ef/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:52.666447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.669850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d4758fe3-9d28-4e01-a84a-21518f5d1619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e26d2c775eb4b019be065da57ce4999'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:52.676628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.689073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1308/2614] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:52 2022] DELETE /identity/v3/users/8e26d2c775eb4b019be065da57ce4999 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:52.690663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc84eb-25b1-4baa-9a0c-e1a8d389f540 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.690946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc84eb-25b1-4baa-9a0c-e1a8d389f540 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.691219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc84eb-25b1-4baa-9a0c-e1a8d389f540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.697221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:52.719962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-64cc84eb-25b1-4baa-9a0c-e1a8d389f540 None None] MFA Rules not processed for user `e11f10329ffc4d2dba60f865786d23ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:52.722045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-64cc84eb-25b1-4baa-9a0c-e1a8d389f540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.749035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c924c76-3be4-4e0f-9c4d-2ecd4872604a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.787170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1309/2615] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:52.788542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.788833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.789108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.810556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:52.812083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.816552 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.817439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.817576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.817801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.818440 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.819946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1310/2616] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:52.820433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.820813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.822073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.834108 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb82dc-f48e-499b-bb3a-8d121a48c247 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.839500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.840141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.840419 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.840696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.841488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.843992 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.850008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1307/2617] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/projects => generated 363 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:52.850849 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.852620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:52.873063 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1311/2618] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:52.874059 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.916079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865a9f3-c3c4-4f20-8268-8518f6bd3300 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.938817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c0b38-3782-4384-a93a-847db082ed30 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.943396 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.944033 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.944314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.944584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.945209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.947121 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.972552 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1312/2619] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:52.972761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.972761 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.973012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.973111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.973111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.980402 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.980595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c0129-e125-42d1-acb7-dae91b2bc802 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:52.981824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.985948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:52.986536 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:52.986818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:52.987090 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:52.987744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:52.989685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:52.992351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1308/2620] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:52.993008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:52.998156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:53.003748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2bf79-c019-4207-9c5c-b4e327c14c6f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.009555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.010274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.010446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.010740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.011376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.013305 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.015576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:53.019026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1313/2621] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:52 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:53.020122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.029970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1309/2622] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:53.031266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.031744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6ac7-064c-415e-89bc-0c0fe198c3fb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.036501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.037150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.037374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.037678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.038331 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.039983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.041466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99f654-7f64-4a92-9df9-6d5c3d88e0d6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.046684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.047367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.047779 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.048060 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] PATH_INFO: `/v3/projects/d22caf128ac34527a28d900015b80290/users/176726a5696645b89541aee67ec227bc/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.053343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1314/2623] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:53.057555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.066258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22caf128ac34527a28d900015b80290, user_id=176726a5696645b89541aee67ec227bc, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.069425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b5767-ea92-4556-a17c-f25a73273fec admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.071597 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.074669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.075437 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.075819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.076272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.077542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.079439 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.092231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1310/2624] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:53 2022] PUT /identity/v3/projects/d22caf128ac34527a28d900015b80290/users/176726a5696645b89541aee67ec227bc/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:53.093511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1315/2625] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:53.097583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.097742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.107962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09330350-43a7-4f79-bdb4-2ee8e40c5ec0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.173532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e1b64-b967-414b-988a-55c5b6b27a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.174824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1b64-b967-414b-988a-55c5b6b27a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.174824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1b64-b967-414b-988a-55c5b6b27a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.175132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1b64-b967-414b-988a-55c5b6b27a99 admin admin] PATH_INFO: `/v3/projects/268fc29a278d4400aa4d93dada90be1e/users/3ee40ac20ffa4b41b887e1d40602d18f/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.178291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4728b-2915-4505-b1c4-82baf38e8ed6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.184058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.184506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.184847 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.185058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.185713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.188216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.193265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e1b64-b967-414b-988a-55c5b6b27a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268fc29a278d4400aa4d93dada90be1e, user_id=3ee40ac20ffa4b41b887e1d40602d18f, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.196669 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e1b64-b967-414b-988a-55c5b6b27a99 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.201425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1311/2626] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:53.204632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.217113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771924-55ab-4551-81e3-6908a2922759 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.219858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1316/2627] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:53 2022] PUT /identity/v3/projects/268fc29a278d4400aa4d93dada90be1e/users/3ee40ac20ffa4b41b887e1d40602d18f/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:53.231561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019f2e-dd21-4f70-919d-310e60f4f665 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.231984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019f2e-dd21-4f70-919d-310e60f4f665 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.232434 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019f2e-dd21-4f70-919d-310e60f4f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.266819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5c019f2e-dd21-4f70-919d-310e60f4f665 None None] MFA Rules not processed for user `3ee40ac20ffa4b41b887e1d40602d18f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:53.268097 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5c019f2e-dd21-4f70-919d-310e60f4f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.291618 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.292314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.293164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.293164 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] PATH_INFO: `/v3/projects/d22caf128ac34527a28d900015b80290/users/176726a5696645b89541aee67ec227bc/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.295034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22caf128ac34527a28d900015b80290, user_id=176726a5696645b89541aee67ec227bc, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.298516 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.311623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1312/2628] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:53 2022] PUT /identity/v3/projects/d22caf128ac34527a28d900015b80290/users/176726a5696645b89541aee67ec227bc/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:53.321800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.323579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.353913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1317/2629] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:53.356449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.356449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.356449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.384227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] MFA Rules not processed for user `176726a5696645b89541aee67ec227bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:53.385447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.394182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4ee2-7700-4dc7-8787-0463a76fd449 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.398928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.399586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.399862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.400142 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] PATH_INFO: `/v3/projects/749eca06a6484ebba440959ae22a5858` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.401407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=749eca06a6484ebba440959ae22a5858)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.403306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.406346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:53.416395 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:53.455839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749eca06a6484ebba440959ae22a5858', 'request_id': 'req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:53.459382 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1318/2630] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:53.460390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.471750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b0056-9506-4f87-8a2c-5c3ffb3d03b6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.471943 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749eca06a6484ebba440959ae22a5858', 'request_id': 'req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:53.478493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749eca06a6484ebba440959ae22a5858', 'request_id': 'req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:53.490842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] The token cache is being invalidate because project 749eca06a6484ebba440959ae22a5858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:53.491229 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:53.492029 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:53.493590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1313/2631] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:53 2022] DELETE /identity/v3/projects/749eca06a6484ebba440959ae22a5858 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:53.494357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.495704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.530835 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8520-3ff2-40c1-a635-490305fe2cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.531447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8520-3ff2-40c1-a635-490305fe2cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.531758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8520-3ff2-40c1-a635-490305fe2cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.532029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8520-3ff2-40c1-a635-490305fe2cf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.532655 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e8520-3ff2-40c1-a635-490305fe2cf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.534572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e8520-3ff2-40c1-a635-490305fe2cf3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.558805 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1570fa8-365f-4e4a-9c3a-97f26ed18e6e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.563445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.564036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.564316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.564589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.566346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.570523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1319/2632] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:53.571884 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b455c-6d74-44c5-8791-7c9ba5dd877c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.572222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b455c-6d74-44c5-8791-7c9ba5dd877c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.572558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b455c-6d74-44c5-8791-7c9ba5dd877c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.592631 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-224b455c-6d74-44c5-8791-7c9ba5dd877c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:53.594533 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-224b455c-6d74-44c5-8791-7c9ba5dd877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.636604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.637883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.651765 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1314/2633] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:53.652942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.655034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.667872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1320/2634] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:53.676389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.676692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.676947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.679840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087f3e4-26b7-4c87-bf5b-5dd53e59b1fb service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.688482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.689198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.689566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.689892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.690754 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.699236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.701746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:53.703752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:53.712152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1321/2635] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:53.716579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.727346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c55c4-b511-41cd-a01c-2f60e8a0f875 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.730660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1315/2636] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/users => generated 299 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:53.732325 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.736244 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.737232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.737771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.738067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.739144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:53.739604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.741982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.756414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b6cec-abc6-469a-8c40-afd6ed5bead3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.760391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1322/2637] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:36:53.762734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.764349 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.764742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.765133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.765458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.766923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.768795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.780022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce6e5a-5f6f-4842-a540-dd42006ad805 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.787077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1316/2638] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:53.788141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.789048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.789681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.790061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.790435 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.791217 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.794811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.800414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658aac-8c13-462b-a6ae-9b119ea75f14 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.806250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.807087 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.807370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.807995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] PATH_INFO: `/v3/projects/658a3a8af66a4e5aa8267b0954e2aafa/users/fdcd6e7a10eb471c8bbfbf0e4caaa9b6/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.822989 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:53.830431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658a3a8af66a4e5aa8267b0954e2aafa, user_id=fdcd6e7a10eb471c8bbfbf0e4caaa9b6, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.831128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1323/2639] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/projects => generated 355 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:53.833743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.835489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.851465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:53.851804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1317/2640] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:53 2022] PUT /identity/v3/projects/658a3a8af66a4e5aa8267b0954e2aafa/users/fdcd6e7a10eb471c8bbfbf0e4caaa9b6/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:53.855898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.919492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0293389-57db-4e21-8a5e-3641833c0cb9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.932673 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e425658-5f66-4480-9bff-7dede5cfaa89 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.939366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.940086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.940843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.941219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.941975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.943958 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.961385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1318/2641] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:53.961385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:53.980433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136df83-e50d-4850-9ba3-954bd3c3db08 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:53.986843 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da364c47-ff96-4794-9e84-d97a89e3f7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.987743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364c47-ff96-4794-9e84-d97a89e3f7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.988293 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364c47-ff96-4794-9e84-d97a89e3f7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.988671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:53.988839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364c47-ff96-4794-9e84-d97a89e3f7e3 admin admin] PATH_INFO: `/v3/projects/658a3a8af66a4e5aa8267b0954e2aafa/users/fdcd6e7a10eb471c8bbfbf0e4caaa9b6/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.990144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:53.990584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:53.991251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da364c47-ff96-4794-9e84-d97a89e3f7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658a3a8af66a4e5aa8267b0954e2aafa, user_id=fdcd6e7a10eb471c8bbfbf0e4caaa9b6, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.991537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:53.992824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:53.994646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da364c47-ff96-4794-9e84-d97a89e3f7e3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:53.995732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.010215 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1319/2642] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:53 2022] PUT /identity/v3/projects/658a3a8af66a4e5aa8267b0954e2aafa/users/fdcd6e7a10eb471c8bbfbf0e4caaa9b6/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:54.021185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.021521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.022035 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.030314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1324/2643] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:53 2022] POST /identity/v3/users => generated 278 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:54.034788 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.051107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e80a7e-c87c-4f93-81ea-1101f866c518 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.059352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] MFA Rules not processed for user `fdcd6e7a10eb471c8bbfbf0e4caaa9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:54.061037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.143202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.143942 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.144227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.144544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.145364 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.147467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.160579 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1325/2644] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:54.164844 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.175387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5577e-b01d-4a22-b825-00f8aa720c7d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.181376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.181937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.182218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.182491 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.183505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.184608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1320/2645] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:54.184947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.199168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.203723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1326/2646] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:54.204721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.207306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.224492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c15e99-231d-4f68-bd3e-75150218cddf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.229767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.230505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.230920 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.231285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] PATH_INFO: `/v3/users/d9065f1aadab4e389bc91ab9f14762e0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.232957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9065f1aadab4e389bc91ab9f14762e0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.235500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.262414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b5f02-090d-4fdb-a6ec-5b616284f63d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.267868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:54.268055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.268763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.268977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.269218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.271331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.282785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:54.300161 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9065f1aadab4e389bc91ab9f14762e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:54.307539 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9065f1aadab4e389bc91ab9f14762e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:54.308252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9065f1aadab4e389bc91ab9f14762e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:54.308886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9065f1aadab4e389bc91ab9f14762e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:54.318600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9065f1aadab4e389bc91ab9f14762e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:54.336298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.337661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ab701-98f0-4fc7-8a46-3985af02af7f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.338377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1327/2647] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:54 2022] DELETE /identity/v3/users/d9065f1aadab4e389bc91ab9f14762e0 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:54.339180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.340606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.347675 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1321/2648] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:54.348919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.349202 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.349478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.371528 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:54.372868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.421174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a9a28-cc41-48bb-9c2e-a8bafa0d1fe0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.426843 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66135e7b-5048-461c-95fd-d98bddd37c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.427417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66135e7b-5048-461c-95fd-d98bddd37c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.427791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66135e7b-5048-461c-95fd-d98bddd37c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.428064 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66135e7b-5048-461c-95fd-d98bddd37c1d admin admin] PATH_INFO: `/v3/projects/10ed6a7a1e4d4f5381217350f1f9f449/users/c19adcec8a404971aa71d3ac6a99e1bd/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.440859 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1322/2649] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:54.444880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66135e7b-5048-461c-95fd-d98bddd37c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ed6a7a1e4d4f5381217350f1f9f449, user_id=c19adcec8a404971aa71d3ac6a99e1bd, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.447828 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66135e7b-5048-461c-95fd-d98bddd37c1d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.450482 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.452680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.463659 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1328/2650] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:54 2022] PUT /identity/v3/projects/10ed6a7a1e4d4f5381217350f1f9f449/users/c19adcec8a404971aa71d3ac6a99e1bd/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:54.475214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.475500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.475911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.509763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] MFA Rules not processed for user `c19adcec8a404971aa71d3ac6a99e1bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:54.511037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:54.536225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fda008-67f9-4425-8f04-31952c3a76be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.604425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.605531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.605900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.606254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.623830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1329/2651] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:54.628767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.652381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:54.673929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:54.720446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045b73-78a7-496f-bff2-466ba84fb271 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.735275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.736283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.736283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.737541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.737974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.742106 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.759192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.760876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.773303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1323/2652] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:54.777696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1330/2653] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:54 2022] POST /identity/v3/projects => generated 357 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:54.781965 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.793244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac31d5-3611-435f-902c-d5a9f68e6e3a service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.856427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.857079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.857382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.857658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.858837 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.860875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.885348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1324/2654] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:54 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:36:54.889225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.901060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29147745-fa2d-45e4-9313-e0a463569cbf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.907034 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.907902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.908160 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.908429 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.909104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.910682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.922370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1331/2655] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:54.926808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.937252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bd91c-ca86-4d45-a15f-188cc4a0791a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:54.941993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:54.943171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:54.943171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:54.943171 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] PATH_INFO: `/v3/projects/98b7bc38b0aa4939acc4631b85f09047/users/5e27fc54e91c4f148050db1987e8b511/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:54.960354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b7bc38b0aa4939acc4631b85f09047, user_id=5e27fc54e91c4f148050db1987e8b511, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:54.963015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:54.986540 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:54.986952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1325/2656] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:54 2022] PUT /identity/v3/projects/98b7bc38b0aa4939acc4631b85f09047/users/5e27fc54e91c4f148050db1987e8b511/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:55.000686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db460230-ebba-47be-864f-49e79696a63f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.071679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.071679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.071679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.071679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.072213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.073830 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.085592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1332/2657] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:55.090231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.100005 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.101330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de29bcb-0614-406e-abe7-2e36f550e440 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.102254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.106103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5cf8e-524b-475b-9acc-ec5e6d250b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.106827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5cf8e-524b-475b-9acc-ec5e6d250b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.107141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5cf8e-524b-475b-9acc-ec5e6d250b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.107307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5cf8e-524b-475b-9acc-ec5e6d250b05 admin admin] PATH_INFO: `/v3/projects/98b7bc38b0aa4939acc4631b85f09047/users/5e27fc54e91c4f148050db1987e8b511/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.109373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5cf8e-524b-475b-9acc-ec5e6d250b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b7bc38b0aa4939acc4631b85f09047, user_id=5e27fc54e91c4f148050db1987e8b511, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.112450 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5cf8e-524b-475b-9acc-ec5e6d250b05 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.120985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444f3cf-d314-4d2c-a3f9-ec222a811761 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.122948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1326/2658] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:55 2022] PUT /identity/v3/projects/98b7bc38b0aa4939acc4631b85f09047/users/5e27fc54e91c4f148050db1987e8b511/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:55.134855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.135135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.135404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.163429 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] MFA Rules not processed for user `5e27fc54e91c4f148050db1987e8b511`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:55.164740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.188918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.189555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.190230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.190510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] PATH_INFO: `/v3/projects/399e4417c8ca47ccaa83f9debb714404` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.191673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] RBAC: Authorizing `identity:delete_project(project_id=399e4417c8ca47ccaa83f9debb714404)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.194013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.226977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:55.227845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:55.238812 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1327/2659] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:36:55.246091 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399e4417c8ca47ccaa83f9debb714404', 'request_id': 'req-b6daea60-7ca1-47b4-ae49-bfb22820606e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.252265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399e4417c8ca47ccaa83f9debb714404', 'request_id': 'req-b6daea60-7ca1-47b4-ae49-bfb22820606e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.252751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.254163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.259982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399e4417c8ca47ccaa83f9debb714404', 'request_id': 'req-b6daea60-7ca1-47b4-ae49-bfb22820606e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.272706 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] The token cache is being invalidate because project 399e4417c8ca47ccaa83f9debb714404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:55.272999 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.273688 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.275486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1333/2660] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:55 2022] DELETE /identity/v3/projects/399e4417c8ca47ccaa83f9debb714404 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:55.279055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.280988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.315158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506adcb1-7a36-4bf7-84f4-97d8fdedec7d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.344970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daea60-7ca1-47b4-ae49-bfb22820606e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.350042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.350677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.350952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.351224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] PATH_INFO: `/v3/users/176726a5696645b89541aee67ec227bc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.352416 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176726a5696645b89541aee67ec227bc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.354365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.364345 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.364888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.365280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.365556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.367487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.403245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176726a5696645b89541aee67ec227bc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.411470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176726a5696645b89541aee67ec227bc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.412282 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176726a5696645b89541aee67ec227bc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.412898 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176726a5696645b89541aee67ec227bc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.420201 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176726a5696645b89541aee67ec227bc'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.436254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.438425 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.441645 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1334/2661] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:36:55 2022] DELETE /identity/v3/users/176726a5696645b89541aee67ec227bc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:55.448179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1328/2662] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:36:55.527421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.529001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.574503 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:55.598557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56ef6f-9999-44e8-94da-91a98f6337d4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.602417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.603039 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.603685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.603685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.605673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.669722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.671727 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83023e44-1586-40c5-9f95-47f3745c02f6 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.680570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1335/2663] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:36:55.729973 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.753731 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e891d-f3a3-49f0-bab9-2ce045ddcb56 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:55.824603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:55.825315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.825661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.826009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] PATH_INFO: `/v3/projects/d22caf128ac34527a28d900015b80290` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.829409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22caf128ac34527a28d900015b80290)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:55.833177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:55.838203 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:55.897086 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22caf128ac34527a28d900015b80290', 'request_id': 'req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.904562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22caf128ac34527a28d900015b80290', 'request_id': 'req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.910701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22caf128ac34527a28d900015b80290', 'request_id': 'req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.922458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] The token cache is being invalidate because project d22caf128ac34527a28d900015b80290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:36:55.922746 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.923388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:36:55.924913 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1329/2664] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:36:55 2022] DELETE /identity/v3/projects/d22caf128ac34527a28d900015b80290 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:55.929848 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8da08-c763-4df5-938c-e97ec8c520b0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:55.930180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8da08-c763-4df5-938c-e97ec8c520b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:55.930453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8da08-c763-4df5-938c-e97ec8c520b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:55.950522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f1f8da08-c763-4df5-938c-e97ec8c520b0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:55.951747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f1f8da08-c763-4df5-938c-e97ec8c520b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:55.966327 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:55.968236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:56.024795 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1336/2665] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:56.032790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.033191 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.033588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.039651 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fb416-0bdd-42fc-810d-4e930ecfcdbb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.043918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.044539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.044822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.045099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.048476 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:56.056809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:56.058012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:56.071692 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1337/2666] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:56.073376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.085267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5f211-5400-47c9-90d4-596c96c1e2c0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.093101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.093720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.094003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.094276 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.094988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:36:56.095968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.098337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.114589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1338/2667] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:36:56.119156 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.125479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.127217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.133134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58140c59-576f-4b44-9f0c-ccaf03279954 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.139890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.140662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.141011 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.141352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.142348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.143532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1330/2668] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:36:56.146397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.202198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1339/2669] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:56 2022] POST /identity/v3/projects => generated 363 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:56.205917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.224246 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc517-8b01-44d7-ad28-e20b5177b08f service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.299354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:56.317533 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.318143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.318474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.318741 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.319389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.321317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.345139 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1331/2670] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:56 2022] POST /identity/v3/users => generated 282 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:56.349811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.361791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaabcba-b18b-4ed4-937e-418447471b4e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.367085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.367747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.368139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.368520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.369290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.371585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.383211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1340/2671] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:56.387763 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.399392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c70d4a-19a7-4d79-b13e-5db425e9f1c2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.404396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b780f1-5884-4618-a19b-da39277127b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.405211 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b780f1-5884-4618-a19b-da39277127b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.405701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b780f1-5884-4618-a19b-da39277127b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.406100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b780f1-5884-4618-a19b-da39277127b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.407389 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b780f1-5884-4618-a19b-da39277127b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.408410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b780f1-5884-4618-a19b-da39277127b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.419578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1332/2672] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:56.423815 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.434691 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445f74-5b2d-4b38-ad9c-d10920065b15 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.440597 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.441265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.441542 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.441819 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] PATH_INFO: `/v3/projects/25cfaa8a40aa40b0a5b8a6d3cbf9d0e2/users/1b933e4f50f840bc906e6c873f5275f8/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.458593 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25cfaa8a40aa40b0a5b8a6d3cbf9d0e2, user_id=1b933e4f50f840bc906e6c873f5275f8, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.461275 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.477062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1341/2673] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:56 2022] PUT /identity/v3/projects/25cfaa8a40aa40b0a5b8a6d3cbf9d0e2/users/1b933e4f50f840bc906e6c873f5275f8/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:56.489452 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.489745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.490062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.523013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] MFA Rules not processed for user `1b933e4f50f840bc906e6c873f5275f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:56.524615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:56.604032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1333/2674] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:56.609410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.624432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:56.676412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fd1df-7e53-4c83-947d-a26b8796531e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.681213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.681811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.682087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.682361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.682984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.684893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.711606 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1342/2675] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:36:56 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:56.715448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.727740 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78334a-59e3-4f09-9d02-781f195aa567 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.749901 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.789067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.789701 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.789988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.790266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.790908 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.792902 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.807872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc3c35-5a94-40de-a1ea-3e43e0ab00dc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.812167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.812807 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.813086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.813365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.817665 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1334/2676] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:36:56 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:36:56.817805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:56.818685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.829012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab57531-1ed9-4560-8560-b8176efbedba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.833684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.834275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.834562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.834842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.835485 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.837111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.848406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1335/2677] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:36:56.852192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.862840 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027c3ff-d03b-4450-870b-05ec45c4e5f7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.870009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:56.871248 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:56.871688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:56.872099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] PATH_INFO: `/v3/projects/b87a3eee2f3d4a009eeaf1890793e327/users/83ffd53f98e545c8ad7948c4aec3ac37/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:56.883450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.884927 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.895952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1343/2678] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:36:56.899766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87a3eee2f3d4a009eeaf1890793e327, user_id=83ffd53f98e545c8ad7948c4aec3ac37, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:56.902542 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:56.917664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1336/2679] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:56 2022] PUT /identity/v3/projects/b87a3eee2f3d4a009eeaf1890793e327/users/83ffd53f98e545c8ad7948c4aec3ac37/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:36:56.921736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:56.931993 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f32ba8-dceb-4170-92f4-ae00f22bcb6d service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:56.997231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:57.015915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddedac-5750-4fa7-9339-5eb4615aa730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:57.016549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddedac-5750-4fa7-9339-5eb4615aa730 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:57.016984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddedac-5750-4fa7-9339-5eb4615aa730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:57.017263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddedac-5750-4fa7-9339-5eb4615aa730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:57.018196 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddedac-5750-4fa7-9339-5eb4615aa730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:57.020407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddedac-5750-4fa7-9339-5eb4615aa730 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:57.034344 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1344/2680] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:36:57.038995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:57.049555 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b59055-5a1c-43c5-84b2-acf23e0e265e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:57.054375 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:57.055026 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:57.055308 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:57.055612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] PATH_INFO: `/v3/projects/b87a3eee2f3d4a009eeaf1890793e327/users/83ffd53f98e545c8ad7948c4aec3ac37/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:57.057549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87a3eee2f3d4a009eeaf1890793e327, user_id=83ffd53f98e545c8ad7948c4aec3ac37, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:57.060235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:57.063556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:36:57.072670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1337/2681] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:36:57 2022] PUT /identity/v3/projects/b87a3eee2f3d4a009eeaf1890793e327/users/83ffd53f98e545c8ad7948c4aec3ac37/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:36:57.084375 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:57.084697 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:57.084981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:57.115611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] MFA Rules not processed for user `83ffd53f98e545c8ad7948c4aec3ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:36:57.116826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:57.187886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1345/2682] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:36:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:36:57.199303 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:57.257441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a22e9-88d6-46fb-aa83-2cf8eaf8d878 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:57.261080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:57.261674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:57.261952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:57.262219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:57.271697 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:57.273110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:57.282445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1338/2683] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:57.358398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:57.368253 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71d635-707b-46f2-a10d-bd855b64cbcc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:57.372057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:57.372661 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:57.372933 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:57.373198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:57.433853 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:57.435285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:57.446159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1346/2684] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:36:58.902367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:36:58.904569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:58.963242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95583-9f50-4ac0-b1ea-41bf4731de83 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:36:58.968132 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:36:58.968747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:36:58.969024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:36:58.969296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:36:58.971025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:36:59.038932 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:36:59.040472 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3994c-4a27-4dd3-80f5-1ddacc968a95 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:36:59.050708 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1339/2685] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:36:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:00.226250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:00.294959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555266a-4d70-4354-a62e-bb046cc0e84e service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:00.300407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:00.301174 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:00.301315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:00.301586 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] PATH_INFO: `/v3/users/1b933e4f50f840bc906e6c873f5275f8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:00.302740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b933e4f50f840bc906e6c873f5275f8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:00.304726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:00.316087 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:00.356444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b933e4f50f840bc906e6c873f5275f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.364290 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b933e4f50f840bc906e6c873f5275f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.364984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b933e4f50f840bc906e6c873f5275f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.365681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b933e4f50f840bc906e6c873f5275f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.372956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b933e4f50f840bc906e6c873f5275f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.394168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1347/2686] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:00 2022] DELETE /identity/v3/users/1b933e4f50f840bc906e6c873f5275f8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:00.398653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:00.399155 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:00.399268 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:00.418323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:00.419860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:00.465147 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:00.481296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1340/2687] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:00.491237 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:00.493162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:00.557817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a3bf-a602-4bdd-91ab-eb918d820d11 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:00.562036 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:00.562647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:00.562925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:00.563198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:00.575839 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:00.577442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:00.587157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1348/2688] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:00.891619 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:00.893157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:00.907419 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beb0bf-d2ae-40f8-8b9a-5d8f3624515d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:00.912475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:00.913835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:00.914129 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:00.914411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] PATH_INFO: `/v3/users/e11f10329ffc4d2dba60f865786d23ef` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:00.915595 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11f10329ffc4d2dba60f865786d23ef)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:00.917521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:00.981134 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11f10329ffc4d2dba60f865786d23ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.989392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11f10329ffc4d2dba60f865786d23ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.990310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11f10329ffc4d2dba60f865786d23ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:00.999169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11f10329ffc4d2dba60f865786d23ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.006967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:01.010340 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.018963 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9d9141-963c-41df-acc3-d8ff5fc2bf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11f10329ffc4d2dba60f865786d23ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.026987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1341/2689] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:00 2022] DELETE /identity/v3/users/e11f10329ffc4d2dba60f865786d23ef => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:01.032359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:01.032646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:01.033430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:01.039743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bf737-d53e-44c0-8159-c559f59ff664 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:01.067061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:01.069567 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.162939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:01.163652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:01.163936 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:01.164265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] PATH_INFO: `/v3/projects/25cfaa8a40aa40b0a5b8a6d3cbf9d0e2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:01.165863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25cfaa8a40aa40b0a5b8a6d3cbf9d0e2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:01.168210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:01.168650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1342/2690] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:01.179954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:01.182218 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.225796 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:01.247643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25cfaa8a40aa40b0a5b8a6d3cbf9d0e2', 'request_id': 'req-309ca241-8893-4d31-9b50-cfce3406146c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.260710 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25cfaa8a40aa40b0a5b8a6d3cbf9d0e2', 'request_id': 'req-309ca241-8893-4d31-9b50-cfce3406146c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.277599 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25cfaa8a40aa40b0a5b8a6d3cbf9d0e2', 'request_id': 'req-309ca241-8893-4d31-9b50-cfce3406146c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.299253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fd4b7-3dd4-429c-af47-5ab6d219d813 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:01.302520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] The token cache is being invalidate because project 25cfaa8a40aa40b0a5b8a6d3cbf9d0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:01.302958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.304171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.308027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1349/2691] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:01 2022] DELETE /identity/v3/projects/25cfaa8a40aa40b0a5b8a6d3cbf9d0e2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:01.312777 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:01.314371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.323221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:01.378846 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:01.379954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:01.379954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:01.380373 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:01.382911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.405551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ca241-8893-4d31-9b50-cfce3406146c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:01.411283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:01.412060 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:01.412410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:01.412753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] PATH_INFO: `/v3/users/83ffd53f98e545c8ad7948c4aec3ac37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:01.414043 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ffd53f98e545c8ad7948c4aec3ac37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:01.416206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:01.467312 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:01.469734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:01.482404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ffd53f98e545c8ad7948c4aec3ac37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.483936 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1343/2692] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:01.492086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ffd53f98e545c8ad7948c4aec3ac37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.492876 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ffd53f98e545c8ad7948c4aec3ac37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.493641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ffd53f98e545c8ad7948c4aec3ac37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.501965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ffd53f98e545c8ad7948c4aec3ac37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:01.521860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1350/2693] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:01 2022] DELETE /identity/v3/users/83ffd53f98e545c8ad7948c4aec3ac37 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:01.862459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:01.864634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.882094 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5b8f1-d882-4004-b245-014f0a0e7480 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:01.923297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:01.924894 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:01.967566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:01.968487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:01.968644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:01.968892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] PATH_INFO: `/v3/projects/05b5c1aa83064941ba8a46fcea497d30` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:01.970047 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b5c1aa83064941ba8a46fcea497d30)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:01.973143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:01.989678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:02.023278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66705dcd-ec0f-4de2-9ab9-2019911e77e7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.034524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b5c1aa83064941ba8a46fcea497d30', 'request_id': 'req-12ec8b18-9bc9-4c25-a959-588351b352e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.042648 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b5c1aa83064941ba8a46fcea497d30', 'request_id': 'req-12ec8b18-9bc9-4c25-a959-588351b352e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.051727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b5c1aa83064941ba8a46fcea497d30', 'request_id': 'req-12ec8b18-9bc9-4c25-a959-588351b352e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.061887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:02.067807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] The token cache is being invalidate because project 05b5c1aa83064941ba8a46fcea497d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:02.068507 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.069899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.072019 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1344/2694] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:01 2022] DELETE /identity/v3/projects/05b5c1aa83064941ba8a46fcea497d30 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:02.077187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.079858 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:02.103896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.104775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.105230 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.105782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] PATH_INFO: `/v3/projects/b87a3eee2f3d4a009eeaf1890793e327` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.106943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87a3eee2f3d4a009eeaf1890793e327)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.110157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.165713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87a3eee2f3d4a009eeaf1890793e327', 'request_id': 'req-0b23001a-7c9f-4747-94a4-0546649d2e96'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.173017 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87a3eee2f3d4a009eeaf1890793e327', 'request_id': 'req-0b23001a-7c9f-4747-94a4-0546649d2e96'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.187735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87a3eee2f3d4a009eeaf1890793e327', 'request_id': 'req-0b23001a-7c9f-4747-94a4-0546649d2e96'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.207735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] The token cache is being invalidate because project b87a3eee2f3d4a009eeaf1890793e327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:02.208026 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.208699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0b23001a-7c9f-4747-94a4-0546649d2e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.210246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1351/2695] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:01 2022] DELETE /identity/v3/projects/b87a3eee2f3d4a009eeaf1890793e327 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:02.215080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f2528-6a2f-4df9-9552-fbe35ea67752 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.215365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f2528-6a2f-4df9-9552-fbe35ea67752 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.215681 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f2528-6a2f-4df9-9552-fbe35ea67752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.236943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-647f2528-6a2f-4df9-9552-fbe35ea67752 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:02.238231 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-647f2528-6a2f-4df9-9552-fbe35ea67752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:02.293150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec8b18-9bc9-4c25-a959-588351b352e1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.308958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1352/2696] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:02.326132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.326420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.326735 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.353192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:02.354449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:02.365636 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1353/2697] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:02.367351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.368020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.368356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.368886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] PATH_INFO: `/v3/users/3ee40ac20ffa4b41b887e1d40602d18f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.370043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee40ac20ffa4b41b887e1d40602d18f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.370574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.372049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.380265 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6537d-34f6-4b8a-9b82-071fd47146d8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.387566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.388456 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.388736 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.389072 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.390721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:02.391300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.393619 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.402715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:02.413377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1354/2698] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:02.414594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.427041 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61452101-48bf-4887-92c2-40aae1b40e79 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.439000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.439000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.439374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.439374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.440050 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.442051 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.445739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee40ac20ffa4b41b887e1d40602d18f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.459724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee40ac20ffa4b41b887e1d40602d18f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.461972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee40ac20ffa4b41b887e1d40602d18f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.471057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee40ac20ffa4b41b887e1d40602d18f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.483125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1355/2699] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:02 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:02.487564 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.489315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:02.490313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee40ac20ffa4b41b887e1d40602d18f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:02.500023 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1345/2700] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:02 2022] DELETE /identity/v3/users/3ee40ac20ffa4b41b887e1d40602d18f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:02.573198 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd1279-7b27-4d4c-90fb-4ff24a324f6c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.661475 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.662571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.663049 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.663561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.664776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.668144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.699326 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1356/2701] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:02 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:02.705024 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.717543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e331d-1d02-40de-b2cd-7039c24ca285 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.729799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.730645 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.730962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.731952 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.732880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.735586 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.746919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1346/2702] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:02.754545 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.776023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972d682-fec4-44f3-98b8-4aeb4b77e4d3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.781374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f32cb-fe5f-4fb7-80fd-5acd213ad3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.782137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f32cb-fe5f-4fb7-80fd-5acd213ad3ea admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.782424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f32cb-fe5f-4fb7-80fd-5acd213ad3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.782902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f32cb-fe5f-4fb7-80fd-5acd213ad3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.783341 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f32cb-fe5f-4fb7-80fd-5acd213ad3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.785107 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f32cb-fe5f-4fb7-80fd-5acd213ad3ea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.808909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1357/2703] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:02.814471 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.827608 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4426-391e-4af5-b929-817581dcaa98 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.833409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.834358 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.834661 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.834934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] PATH_INFO: `/v3/projects/fdb56f5efaac4526bff1d2f51c04937d/users/d170453e667b47e7a62a234dfef269f2/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.848712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:02.853624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb56f5efaac4526bff1d2f51c04937d, user_id=d170453e667b47e7a62a234dfef269f2, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:02.856531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:02.877736 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1347/2704] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:02 2022] PUT /identity/v3/projects/fdb56f5efaac4526bff1d2f51c04937d/users/d170453e667b47e7a62a234dfef269f2/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:02.889303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.890013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.890351 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:02.902955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:02.905686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:02.957261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] MFA Rules not processed for user `d170453e667b47e7a62a234dfef269f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:02.958525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:02.990379 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f59c-0e82-4a55-8adf-c65e78ced1b1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:02.996504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:02.997237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:02.997587 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:02.997923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] PATH_INFO: `/v3/projects/268fc29a278d4400aa4d93dada90be1e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.000546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268fc29a278d4400aa4d93dada90be1e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.002459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.027510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:03.052803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1358/2705] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:03.057289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.064072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268fc29a278d4400aa4d93dada90be1e', 'request_id': 'req-ca2bf6a7-05ef-4348-bc74-fd72c5315718'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.069694 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23267-738f-4a3b-8347-ce623caf8afc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.074434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268fc29a278d4400aa4d93dada90be1e', 'request_id': 'req-ca2bf6a7-05ef-4348-bc74-fd72c5315718'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.081422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268fc29a278d4400aa4d93dada90be1e', 'request_id': 'req-ca2bf6a7-05ef-4348-bc74-fd72c5315718'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.094102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] The token cache is being invalidate because project 268fc29a278d4400aa4d93dada90be1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:03.094393 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.095599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.097361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1348/2706] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:02 2022] DELETE /identity/v3/projects/268fc29a278d4400aa4d93dada90be1e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:03.101222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.102539 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.143474 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.144126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.144407 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.144682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.145596 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.147272 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.168275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bf6a7-05ef-4348-bc74-fd72c5315718 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.173836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.174468 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.174747 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.175021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] PATH_INFO: `/v3/users/fdcd6e7a10eb471c8bbfbf0e4caaa9b6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.175922 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1359/2707] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:03.176184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdcd6e7a10eb471c8bbfbf0e4caaa9b6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.178190 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.180428 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.181997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.201399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:03.229382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdcd6e7a10eb471c8bbfbf0e4caaa9b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.237691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdcd6e7a10eb471c8bbfbf0e4caaa9b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.238377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdcd6e7a10eb471c8bbfbf0e4caaa9b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.245762 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdcd6e7a10eb471c8bbfbf0e4caaa9b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.251269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf5de9-688b-439f-9511-e99471196fa8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.255965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.256577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.256858 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.257135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.257766 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.259136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdcd6e7a10eb471c8bbfbf0e4caaa9b6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.259718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.265331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1349/2708] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:03 2022] DELETE /identity/v3/users/fdcd6e7a10eb471c8bbfbf0e4caaa9b6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:03.284541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1360/2709] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:03.288724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.290611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.361018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d82b-4a5d-47e7-bfdd-67d81b290a0f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.365674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.366287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.366565 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.366833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.367509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.369167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.381694 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1350/2710] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:03.386186 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.397598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4fb64-61f3-4bae-ac7c-a909ac43df69 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.402483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.403208 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.403498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.403811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] PATH_INFO: `/v3/projects/c022380c919c45888e8bc7d5878d1d97/users/c64e33b438a741b899a58fdcdbe19b03/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.430209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c022380c919c45888e8bc7d5878d1d97, user_id=c64e33b438a741b899a58fdcdbe19b03, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.432907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.449837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1361/2711] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:03 2022] PUT /identity/v3/projects/c022380c919c45888e8bc7d5878d1d97/users/c64e33b438a741b899a58fdcdbe19b03/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:03.453872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.464097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aed4de-5cd3-4c16-8408-a7e541971caa admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.529381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.530573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.530852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.531123 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.531848 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.533449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.544868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1351/2712] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:03.549761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.560647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4ae6e-1b31-48fd-8af2-1271dc2d2aea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.569073 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.569573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2484cfa-1a65-4f7f-884d-adae0f63bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.570322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2484cfa-1a65-4f7f-884d-adae0f63bbb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.570610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2484cfa-1a65-4f7f-884d-adae0f63bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.571801 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2484cfa-1a65-4f7f-884d-adae0f63bbb5 admin admin] PATH_INFO: `/v3/projects/c022380c919c45888e8bc7d5878d1d97/users/c64e33b438a741b899a58fdcdbe19b03/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.573010 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.573975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2484cfa-1a65-4f7f-884d-adae0f63bbb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c022380c919c45888e8bc7d5878d1d97, user_id=c64e33b438a741b899a58fdcdbe19b03, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.577224 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2484cfa-1a65-4f7f-884d-adae0f63bbb5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.589712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d1ee-761d-4765-9a65-3af44634be79 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.590139 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1362/2713] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:03 2022] PUT /identity/v3/projects/c022380c919c45888e8bc7d5878d1d97/users/c64e33b438a741b899a58fdcdbe19b03/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:03.602957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.603577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.603713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.624891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:03.637094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] MFA Rules not processed for user `c64e33b438a741b899a58fdcdbe19b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:03.638385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.656956 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.657625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.657910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.658021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:03.658189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] PATH_INFO: `/v3/projects/658a3a8af66a4e5aa8267b0954e2aafa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.659515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658a3a8af66a4e5aa8267b0954e2aafa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.661605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.709511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658a3a8af66a4e5aa8267b0954e2aafa', 'request_id': 'req-12f7091f-c578-40c5-8ac3-539e3bf33c41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.711991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1363/2714] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:03.720461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658a3a8af66a4e5aa8267b0954e2aafa', 'request_id': 'req-12f7091f-c578-40c5-8ac3-539e3bf33c41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.724149 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.726058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.726991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658a3a8af66a4e5aa8267b0954e2aafa', 'request_id': 'req-12f7091f-c578-40c5-8ac3-539e3bf33c41'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.738641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] The token cache is being invalidate because project 658a3a8af66a4e5aa8267b0954e2aafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:03.739048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.739889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-12f7091f-c578-40c5-8ac3-539e3bf33c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:03.741376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1352/2715] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:03 2022] DELETE /identity/v3/projects/658a3a8af66a4e5aa8267b0954e2aafa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:03.745976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c19665-f272-40e5-8780-35a25fe07378 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.746247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c19665-f272-40e5-8780-35a25fe07378 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.746513 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c19665-f272-40e5-8780-35a25fe07378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.766377 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e4c19665-f272-40e5-8780-35a25fe07378 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:03.767596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4c19665-f272-40e5-8780-35a25fe07378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.786053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0cedb-ec24-485f-9981-0b12789194df None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.832286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1353/2716] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:03.841124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.841307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.841740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.841900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.842008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.842124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.842332 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.844793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.862179 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:03.863366 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:03.871223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1354/2717] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:03.875511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.884395 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1c09-62f6-4dd8-8ff3-f804f3752320 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.891069 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.891538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.891839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.892107 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.892814 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:03.893120 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.895223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.907480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1355/2718] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:37:03.911739 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.917079 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.918544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.921420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34d49-a8b7-4895-b26c-4e076e3b3fe8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:03.925976 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:03.926558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:03.926842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:03.927119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:03.927772 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:03.928523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1364/2719] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:37:03.929638 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:03.968689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1356/2720] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:03.972962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:03.986135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de198-798d-4e4a-8f3d-fe362ebb758a service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:04.012134 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:04.073285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:04.073899 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:04.074182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:04.074996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:04.075135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:04.077185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:04.103820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1365/2721] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:03 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:04.104969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:04.117251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a401b8e-1c73-4cc8-b474-537567a6580c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:04.121615 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:04.122954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:04.122954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:04.122954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:04.123417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:04.125085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:04.138136 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1357/2722] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:37:04.142671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:04.155294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68a07-8fd8-462f-9cf3-5da15b438832 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:04.161708 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aa277-527d-4578-b00e-fe824385a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:04.162399 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa277-527d-4578-b00e-fe824385a73c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:04.162677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa277-527d-4578-b00e-fe824385a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:04.162947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aa277-527d-4578-b00e-fe824385a73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:04.163732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa277-527d-4578-b00e-fe824385a73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:04.165548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aa277-527d-4578-b00e-fe824385a73c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:04.179580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1366/2723] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:37:04.183075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:04.193903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9139c-8df7-43e1-b4c1-f4bf898bc739 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:04.199500 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:04.200287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:04.200631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:04.201052 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] PATH_INFO: `/v3/projects/b5ecb76de8c443ba93d919ab2bad03c2/users/d8de3e1c91e343d6924f7f4ce08bf4e0/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:04.205493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:04.220764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ecb76de8c443ba93d919ab2bad03c2, user_id=d8de3e1c91e343d6924f7f4ce08bf4e0, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:04.224582 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:04.242743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1358/2724] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:04 2022] PUT /identity/v3/projects/b5ecb76de8c443ba93d919ab2bad03c2/users/d8de3e1c91e343d6924f7f4ce08bf4e0/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:04.248311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e870098-1b85-47ad-8ca5-6964709bc64f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:04.248622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e870098-1b85-47ad-8ca5-6964709bc64f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:04.248904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e870098-1b85-47ad-8ca5-6964709bc64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:04.269906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2e870098-1b85-47ad-8ca5-6964709bc64f None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:04.271113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2e870098-1b85-47ad-8ca5-6964709bc64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:04.356140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1367/2725] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:04.366854 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:04.374250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:04.457146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fd274-d7a0-4f00-a737-2a3dd371a00b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:04.461084 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:04.461688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:04.461964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:04.462249 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:04.472827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:04.474444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:04.484435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1359/2726] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:05.134845 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:05.135300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:05.135444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:05.169638 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] MFA Rules not processed for user `d8de3e1c91e343d6924f7f4ce08bf4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:05.171180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:05.250943 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1368/2727] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:05.262942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:05.265157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:05.325660 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c2a9e-402b-4f89-a79c-5a520221899c service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:05.329643 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf62c4c-8a1b-472e-9ea0-f37a0884ceb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:05.330253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf62c4c-8a1b-472e-9ea0-f37a0884ceb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:05.330529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf62c4c-8a1b-472e-9ea0-f37a0884ceb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:05.330948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf62c4c-8a1b-472e-9ea0-f37a0884ceb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:05.340523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf62c4c-8a1b-472e-9ea0-f37a0884ceb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:05.341982 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf62c4c-8a1b-472e-9ea0-f37a0884ceb4 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:05.352700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1360/2728] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:05.512530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:05.515252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:05.529510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213da092-677a-4df0-9c4b-3e01f8aefb57 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:05.536903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:05.537967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:05.538534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:05.539016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] PATH_INFO: `/v3/users/d170453e667b47e7a62a234dfef269f2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:05.540424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d170453e667b47e7a62a234dfef269f2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:05.542412 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:05.570852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:05.605410 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd170453e667b47e7a62a234dfef269f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:05.621610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd170453e667b47e7a62a234dfef269f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:05.622374 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd170453e667b47e7a62a234dfef269f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:05.625117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd170453e667b47e7a62a234dfef269f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:05.636180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd170453e667b47e7a62a234dfef269f2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:05.667772 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1369/2729] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:05 2022] DELETE /identity/v3/users/d170453e667b47e7a62a234dfef269f2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:05.672944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:05.674264 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:05.674900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:05.704122 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:05.705549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:05.769766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1361/2730] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:05.779698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:05.781717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:05.862939 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2441e0a-0b77-4e75-8ac0-d817c89f9b91 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:05.867118 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:05.867846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:05.868104 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:05.868450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:05.880598 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:05.882168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:05.892188 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1370/2731] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:06.182341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:06.183884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:06.196501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39ecbe-6337-4a99-9bb6-0e9812a662e7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:06.201105 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:06.201917 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:06.202186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:06.202460 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] PATH_INFO: `/v3/projects/fdb56f5efaac4526bff1d2f51c04937d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:06.203880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb56f5efaac4526bff1d2f51c04937d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:06.205492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:06.237089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:06.254380 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb56f5efaac4526bff1d2f51c04937d', 'request_id': 'req-09457cb0-4b56-4201-b74b-a83098389b75'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.261556 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb56f5efaac4526bff1d2f51c04937d', 'request_id': 'req-09457cb0-4b56-4201-b74b-a83098389b75'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.267197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb56f5efaac4526bff1d2f51c04937d', 'request_id': 'req-09457cb0-4b56-4201-b74b-a83098389b75'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.278181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] The token cache is being invalidate because project fdb56f5efaac4526bff1d2f51c04937d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:06.278474 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.279109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.281174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1362/2732] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:06 2022] DELETE /identity/v3/projects/fdb56f5efaac4526bff1d2f51c04937d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:06.285270 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:06.294862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8febf23-61ef-4db0-9f81-4f0af6ad9ec6 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:06.352852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:06.353477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:06.353758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:06.354030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] PATH_INFO: `/v3/users/c64e33b438a741b899a58fdcdbe19b03` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:06.355141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64e33b438a741b899a58fdcdbe19b03)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:06.357092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:06.401285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:06.402902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64e33b438a741b899a58fdcdbe19b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.410980 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64e33b438a741b899a58fdcdbe19b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.411886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64e33b438a741b899a58fdcdbe19b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.412625 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64e33b438a741b899a58fdcdbe19b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.419476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c58cf005-68f4-4194-8d42-f4bfd446dd47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64e33b438a741b899a58fdcdbe19b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.437052 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1371/2733] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:06 2022] DELETE /identity/v3/users/c64e33b438a741b899a58fdcdbe19b03 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:06.709558 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:06.710974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:06.724944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09457cb0-4b56-4201-b74b-a83098389b75 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:06.762101 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047742a-8b63-4c15-adbf-b2e28894b508 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:06.762546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047742a-8b63-4c15-adbf-b2e28894b508 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:06.762817 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047742a-8b63-4c15-adbf-b2e28894b508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:06.782709 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c047742a-8b63-4c15-adbf-b2e28894b508 None None] MFA Rules not processed for user `d8de3e1c91e343d6924f7f4ce08bf4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:06.783925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c047742a-8b63-4c15-adbf-b2e28894b508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:06.787121 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:06.787728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:06.788045 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:06.788319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] PATH_INFO: `/v3/projects/c022380c919c45888e8bc7d5878d1d97` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:06.789435 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c022380c919c45888e8bc7d5878d1d97)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:06.791350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:06.840018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1372/2734] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:06.845598 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c022380c919c45888e8bc7d5878d1d97', 'request_id': 'req-78044936-1f0b-4157-a17a-84579b5673a1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.855408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c022380c919c45888e8bc7d5878d1d97', 'request_id': 'req-78044936-1f0b-4157-a17a-84579b5673a1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.862183 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c022380c919c45888e8bc7d5878d1d97', 'request_id': 'req-78044936-1f0b-4157-a17a-84579b5673a1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.873719 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] The token cache is being invalidate because project c022380c919c45888e8bc7d5878d1d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:06.874011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.874789 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78044936-1f0b-4157-a17a-84579b5673a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:06.876267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1363/2735] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:06 2022] DELETE /identity/v3/projects/c022380c919c45888e8bc7d5878d1d97 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:06.881311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:06.881595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:06.881868 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:06.905947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:06.907298 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:06.983719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1373/2736] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:07.005116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.006299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.006576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.027032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:07.028276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:07.037871 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1364/2737] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:07.042347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.056219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802f5df-bedb-4ab3-a629-da256ac3d8c3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.073095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.073740 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.074002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.074496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.075162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:07.075467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.077459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.097856 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1374/2738] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:07.099274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.111280 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bba7e-e284-45c2-bb04-f6fa3a3650b0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.116427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.117027 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.117401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.117678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.118338 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.119578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.120720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.121347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:07.154014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1365/2739] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:07.155824 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.173758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:07.182345 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164321f9-8af5-4079-99c6-16fcf3f0d5e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.190969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:07.223208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223efc6c-ee57-4be8-9f89-428835ea809d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.228611 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.229330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.229785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.230079 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.231075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.233083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.234718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.235329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.235639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.235918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.237671 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:07.247832 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.249916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.258584 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1366/2740] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:07.258780 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1375/2741] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:37:07.259659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.270363 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428bbd5-a036-4b75-b331-c79d8cbd1989 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.275113 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.275756 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.276042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.276313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.276975 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.278614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.288889 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1367/2742] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:37:07.292862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.305254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee83a25-2d51-4ff4-a02f-e6448dc72e79 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.309938 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84163fa-7263-4b89-a494-23414d72b1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.310531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84163fa-7263-4b89-a494-23414d72b1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.310852 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84163fa-7263-4b89-a494-23414d72b1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.311122 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84163fa-7263-4b89-a494-23414d72b1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.311785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84163fa-7263-4b89-a494-23414d72b1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.313361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84163fa-7263-4b89-a494-23414d72b1ce admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.327814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1376/2743] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:07.329355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.339381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f072d9-2455-46fa-a887-6a43d637b64d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.344738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.345388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.345670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.345948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] PATH_INFO: `/v3/projects/06b7eb5b1f414d18b53933d6d7272129/users/13fc8e04675345a7a40500f9a6270d7c/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.361669 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b7eb5b1f414d18b53933d6d7272129, user_id=13fc8e04675345a7a40500f9a6270d7c, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.364341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.378001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1368/2744] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:07 2022] PUT /identity/v3/projects/06b7eb5b1f414d18b53933d6d7272129/users/13fc8e04675345a7a40500f9a6270d7c/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:07.388209 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.388501 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.388774 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.418622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] MFA Rules not processed for user `13fc8e04675345a7a40500f9a6270d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:07.419864 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:07.486720 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1377/2745] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:07.491417 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.562700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770dc28-d8b9-4391-8ee8-1525a848363a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.567591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.568316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.568723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.569100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.570753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.573009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.607527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1369/2746] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:07.612241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.624222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87654c-ca02-44ab-88bf-775874e8a4db None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.686289 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.687180 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.687530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.687896 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.688510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.690483 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.711981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1378/2747] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:07.715785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.727807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb03a-9072-4599-a8df-55e792ff8add admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.732495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.733742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.734008 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.734279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.734924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.736538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.747025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1370/2748] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:07.751444 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.762175 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3654bdf-67ed-4f89-a516-aa5e8b88e89c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.767711 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.768372 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.768649 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.768923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] PATH_INFO: `/v3/projects/805a5ad4dbcf4d7f9f24f89900aac358/users/5e99d8347816427e8dd05cc5654e525c/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.785309 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805a5ad4dbcf4d7f9f24f89900aac358, user_id=5e99d8347816427e8dd05cc5654e525c, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.788042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.796804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:07.803363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1379/2749] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:07 2022] PUT /identity/v3/projects/805a5ad4dbcf4d7f9f24f89900aac358/users/5e99d8347816427e8dd05cc5654e525c/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:07.809118 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.820391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28855b-c4ed-4041-9f68-7d6c0c48f502 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.864111 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:07.881144 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1936280-328f-4cc9-b11c-d3f84b412d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.881758 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1936280-328f-4cc9-b11c-d3f84b412d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.882032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1936280-328f-4cc9-b11c-d3f84b412d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.882309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1936280-328f-4cc9-b11c-d3f84b412d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.883053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1936280-328f-4cc9-b11c-d3f84b412d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.884672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1936280-328f-4cc9-b11c-d3f84b412d88 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.896829 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1371/2750] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:07.901392 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:07.913154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f88725-aab3-4231-8a63-033a50ae2a50 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:07.917908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:07.918662 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.919002 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.919284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] PATH_INFO: `/v3/projects/805a5ad4dbcf4d7f9f24f89900aac358/users/5e99d8347816427e8dd05cc5654e525c/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.921842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=805a5ad4dbcf4d7f9f24f89900aac358, user_id=5e99d8347816427e8dd05cc5654e525c, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:07.924504 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:07.935365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1380/2751] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:07 2022] PUT /identity/v3/projects/805a5ad4dbcf4d7f9f24f89900aac358/users/5e99d8347816427e8dd05cc5654e525c/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:07.945834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:07.946110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:07.946405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:07.977037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] MFA Rules not processed for user `5e99d8347816427e8dd05cc5654e525c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:07.978266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:08.054868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1372/2752] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:08.068561 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:08.070252 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:08.133611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea0129-c163-49c5-9bba-66664ea44ebf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:08.137581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:08.137964 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:08.138236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:08.138499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:08.150948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:08.151367 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:08.163462 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1381/2753] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:08.247672 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:08.250447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:08.263216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f333e-ee2f-4544-8208-775b82ff0818 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:08.269104 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefaf17-cf82-485e-83a5-88b5f3bf1597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:08.269721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefaf17-cf82-485e-83a5-88b5f3bf1597 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:08.269999 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefaf17-cf82-485e-83a5-88b5f3bf1597 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:08.270296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefaf17-cf82-485e-83a5-88b5f3bf1597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:08.335998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefaf17-cf82-485e-83a5-88b5f3bf1597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:08.337580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefaf17-cf82-485e-83a5-88b5f3bf1597 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:08.347752 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1373/2754] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:37:08.621840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:08.752477 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dea6a-6028-41b3-aa93-e5b47585f701 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:08.758432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:08.759070 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:08.759571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:08.759860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] PATH_INFO: `/v3/users/13fc8e04675345a7a40500f9a6270d7c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:08.761273 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13fc8e04675345a7a40500f9a6270d7c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:08.763166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:08.832166 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13fc8e04675345a7a40500f9a6270d7c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:08.833931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:08.841376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13fc8e04675345a7a40500f9a6270d7c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:08.842831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13fc8e04675345a7a40500f9a6270d7c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:08.843909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13fc8e04675345a7a40500f9a6270d7c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:08.852725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13fc8e04675345a7a40500f9a6270d7c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:08.872310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1382/2755] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:08 2022] DELETE /identity/v3/users/13fc8e04675345a7a40500f9a6270d7c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:08.877385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:08.877745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:08.878092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:08.931020 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:08.932309 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:08.966351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:08.995455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1374/2756] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:09.006279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:09.007724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:09.085297 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113290-34c1-4855-88bc-f05b625320e9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:09.088845 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:09.089451 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:09.089721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:09.089996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:09.099730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:09.101248 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:09.111902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1383/2757] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:09.409692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:09.411252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:09.438901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bf43a-18d4-4695-8721-7bb9a139e308 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:09.445929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:09.446649 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:09.447001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:09.447344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] PATH_INFO: `/v3/projects/06b7eb5b1f414d18b53933d6d7272129` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:09.448596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b7eb5b1f414d18b53933d6d7272129)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:09.451365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:09.504430 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b7eb5b1f414d18b53933d6d7272129', 'request_id': 'req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.513295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b7eb5b1f414d18b53933d6d7272129', 'request_id': 'req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.520596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b7eb5b1f414d18b53933d6d7272129', 'request_id': 'req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.533691 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] The token cache is being invalidate because project 06b7eb5b1f414d18b53933d6d7272129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:09.534034 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.534996 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.538706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1375/2758] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:09 2022] DELETE /identity/v3/projects/06b7eb5b1f414d18b53933d6d7272129 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:09.544588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:09.560184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4559672-70ca-4e75-a655-0759cb320799 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:09.651683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:09.652485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:09.652919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:09.653502 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] PATH_INFO: `/v3/users/5e99d8347816427e8dd05cc5654e525c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:09.654380 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e99d8347816427e8dd05cc5654e525c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:09.656577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:09.725442 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e99d8347816427e8dd05cc5654e525c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.735581 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e99d8347816427e8dd05cc5654e525c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.736841 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e99d8347816427e8dd05cc5654e525c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.740164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e99d8347816427e8dd05cc5654e525c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.742021 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:09.753137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3d5bf7b4-4d6a-4187-a9ec-4d554fe74b13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e99d8347816427e8dd05cc5654e525c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:09.779918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1384/2759] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:09 2022] DELETE /identity/v3/users/5e99d8347816427e8dd05cc5654e525c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:10.393099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:10.398317 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:10.419721 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b3ac3-3bdd-4ca5-8b79-a02da32ed4d0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:10.501781 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:10.502801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:10.502972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:10.503580 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] PATH_INFO: `/v3/projects/805a5ad4dbcf4d7f9f24f89900aac358` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:10.505089 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805a5ad4dbcf4d7f9f24f89900aac358)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:10.507149 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:10.576931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805a5ad4dbcf4d7f9f24f89900aac358', 'request_id': 'req-b59c34f9-5414-4ed3-b6ec-346cf98deb58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:10.585955 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805a5ad4dbcf4d7f9f24f89900aac358', 'request_id': 'req-b59c34f9-5414-4ed3-b6ec-346cf98deb58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:10.592732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:10.594332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805a5ad4dbcf4d7f9f24f89900aac358', 'request_id': 'req-b59c34f9-5414-4ed3-b6ec-346cf98deb58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:10.608959 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] The token cache is being invalidate because project 805a5ad4dbcf4d7f9f24f89900aac358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:10.609257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:10.609931 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b59c34f9-5414-4ed3-b6ec-346cf98deb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:10.611614 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1376/2760] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:10 2022] DELETE /identity/v3/projects/805a5ad4dbcf4d7f9f24f89900aac358 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:10.616499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:10.616836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:10.617095 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:10.684408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:10.685800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:10.773776 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1385/2761] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:10.782880 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:10.783292 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:10.783878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:10.826119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:10.828817 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:10.840572 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1377/2762] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:10.845086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:10.855318 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cf67d-5c30-4f19-a99f-d5ad8a8fdaeb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:10.860644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:10.861333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:10.861683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:10.862030 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:10.862753 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:10.863062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:10.864698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:10.877485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1386/2763] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:10.882174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:10.892231 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860afbb9-35e8-4816-bb7e-719a0e8c74b6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:10.896970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:10.897647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:10.897998 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:10.898359 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:10.899075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:10.901175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:10.934250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1378/2764] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:10 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:10.939956 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:10.951012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13655b-a455-48e0-9016-833952471485 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.014947 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.015612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.015900 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.016182 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.016829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.018757 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.044227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1387/2765] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:10 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:11.046438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:11.060068 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7423cc0-f277-4724-8023-489cafb219e8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.067757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.067757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.067970 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.068159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.068922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.070776 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.087131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1379/2766] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:11.088628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:11.102035 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111942e-cdf9-42ba-977d-e2644af6785e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.108557 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc6b04-1235-464b-98b1-c697ff19a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.109791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc6b04-1235-464b-98b1-c697ff19a0ca admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.110239 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc6b04-1235-464b-98b1-c697ff19a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.110673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc6b04-1235-464b-98b1-c697ff19a0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.111499 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc6b04-1235-464b-98b1-c697ff19a0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.113719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc6b04-1235-464b-98b1-c697ff19a0ca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.128315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1388/2767] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:11.132650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:11.145870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af8f5f-d70e-4293-a41a-f8af07da8f1c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.156049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.157692 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.158141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.158626 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] PATH_INFO: `/v3/projects/010d0fa80bd546668b6d6de9adf8c02c/users/ff5d072b1e084b87aadff728b30fb33c/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.177981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=010d0fa80bd546668b6d6de9adf8c02c, user_id=ff5d072b1e084b87aadff728b30fb33c, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.184186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.200314 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1380/2768] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:11 2022] PUT /identity/v3/projects/010d0fa80bd546668b6d6de9adf8c02c/users/ff5d072b1e084b87aadff728b30fb33c/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:11.206575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.206575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.207179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.231127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:11.232982 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:11.260111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:11.330556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1389/2769] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:11.343232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:11.345013 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:11.377555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:11.379133 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:11.394610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cced4dc-c0a0-4aae-9757-db53da087a43 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.399969 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.400787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.401125 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.401467 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] PATH_INFO: `/v3/users/c19adcec8a404971aa71d3ac6a99e1bd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.402650 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19adcec8a404971aa71d3ac6a99e1bd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.404861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.419760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e7b8-4d8b-46c5-a024-96145a993524 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.426388 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482108-e7a1-4e8f-a0e7-a2a71ecbe1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.427238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482108-e7a1-4e8f-a0e7-a2a71ecbe1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.427742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482108-e7a1-4e8f-a0e7-a2a71ecbe1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.428145 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482108-e7a1-4e8f-a0e7-a2a71ecbe1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.443198 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482108-e7a1-4e8f-a0e7-a2a71ecbe1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.445242 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482108-e7a1-4e8f-a0e7-a2a71ecbe1d8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.456920 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1381/2770] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:11.458714 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19adcec8a404971aa71d3ac6a99e1bd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:11.468857 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19adcec8a404971aa71d3ac6a99e1bd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:11.469791 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19adcec8a404971aa71d3ac6a99e1bd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:11.470551 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19adcec8a404971aa71d3ac6a99e1bd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:11.479696 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19adcec8a404971aa71d3ac6a99e1bd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:11.500260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1390/2771] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:11 2022] DELETE /identity/v3/users/c19adcec8a404971aa71d3ac6a99e1bd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:11.505017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.505367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.505713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.525818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:11.527022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:11.599857 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1382/2772] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:11.612120 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:11.613549 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:11.692102 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ae4be-147d-4edc-bd07-4579330fb807 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:11.695952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:11.696588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:11.696855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:11.697132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:11.707083 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:11.708560 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:11.719873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1391/2773] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:12.124116 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.125865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:12.149320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba72ca-e9b1-4e1b-a31f-4ba15f79700a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.156257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.157005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.157385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.157724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] PATH_INFO: `/v3/projects/10ed6a7a1e4d4f5381217350f1f9f449` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.158944 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ed6a7a1e4d4f5381217350f1f9f449)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.161018 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.198561 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:12.223974 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ed6a7a1e4d4f5381217350f1f9f449', 'request_id': 'req-23514020-916c-426d-bd05-0c172880ec81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.232322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ed6a7a1e4d4f5381217350f1f9f449', 'request_id': 'req-23514020-916c-426d-bd05-0c172880ec81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.239483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ed6a7a1e4d4f5381217350f1f9f449', 'request_id': 'req-23514020-916c-426d-bd05-0c172880ec81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.252888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] The token cache is being invalidate because project 10ed6a7a1e4d4f5381217350f1f9f449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:12.253039 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.253659 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-23514020-916c-426d-bd05-0c172880ec81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.255686 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1383/2774] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:12 2022] DELETE /identity/v3/projects/10ed6a7a1e4d4f5381217350f1f9f449 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:12.259313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.271008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d98dee-b054-4c5f-a513-7ed5f2dd6260 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.301519 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:12.347713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e65e-0832-4129-955a-bf034093a386 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.348066 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e65e-0832-4129-955a-bf034093a386 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.348267 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e65e-0832-4129-955a-bf034093a386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.350304 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.350961 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.351243 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.351520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] PATH_INFO: `/v3/users/5e27fc54e91c4f148050db1987e8b511` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.352775 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e27fc54e91c4f148050db1987e8b511)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.354840 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.383458 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4726e65e-0832-4129-955a-bf034093a386 None None] MFA Rules not processed for user `ff5d072b1e084b87aadff728b30fb33c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:12.384901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4726e65e-0832-4129-955a-bf034093a386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:12.417962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e27fc54e91c4f148050db1987e8b511'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.432218 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e27fc54e91c4f148050db1987e8b511'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.433594 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e27fc54e91c4f148050db1987e8b511'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.434465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e27fc54e91c4f148050db1987e8b511'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.445923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e27fc54e91c4f148050db1987e8b511'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.470335 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1392/2775] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:12 2022] DELETE /identity/v3/users/5e27fc54e91c4f148050db1987e8b511 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:12.501911 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1384/2776] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:12.506792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.508274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:12.594226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb8050-d054-4f43-8ab9-220a4b5975d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.601127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.602827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.603445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.603903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.604767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.608138 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.657421 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1393/2777] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:12 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:12.659163 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726e65e-0832-4129-955a-bf034093a386 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.675012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726e65e-0832-4129-955a-bf034093a386 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.772612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.773237 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.773521 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.773798 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.775244 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.777226 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.810793 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1385/2778] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:12 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:12.815236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.826937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2707ee-f3ee-47cf-9558-f251d5887deb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.832212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.832911 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.833261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.833663 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.834389 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.836640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.853668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.855029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1394/2779] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:37:12.856371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:12.859177 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.870886 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d5704-3a3b-4b62-9d24-965e2e4dd929 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.874300 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdc738-f16d-45f1-bbd6-44d2f0d9ba75 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.876177 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.876994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.877276 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.877548 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] PATH_INFO: `/v3/projects/98b7bc38b0aa4939acc4631b85f09047` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.878723 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b7bc38b0aa4939acc4631b85f09047)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.880147 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:12.880650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.881044 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.881213 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.881466 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] PATH_INFO: `/v3/projects/5f82be918c6d4b5d8067f95d665042cd/users/9d09e356b48940e4a21ec62f978efa3c/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.899388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f82be918c6d4b5d8067f95d665042cd, user_id=9d09e356b48940e4a21ec62f978efa3c, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:12.902197 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:12.912543 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:12.917247 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:12.919626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1395/2780] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:12 2022] PUT /identity/v3/projects/5f82be918c6d4b5d8067f95d665042cd/users/9d09e356b48940e4a21ec62f978efa3c/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:12.924378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:12.939894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b7bc38b0aa4939acc4631b85f09047', 'request_id': 'req-f504607c-52ef-4827-9fbb-20cbeac67532'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.948290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b7bc38b0aa4939acc4631b85f09047', 'request_id': 'req-f504607c-52ef-4827-9fbb-20cbeac67532'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.955409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b7bc38b0aa4939acc4631b85f09047', 'request_id': 'req-f504607c-52ef-4827-9fbb-20cbeac67532'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.967284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] The token cache is being invalidate because project 98b7bc38b0aa4939acc4631b85f09047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:12.967593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.968372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f504607c-52ef-4827-9fbb-20cbeac67532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:12.969861 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1386/2781] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:12 2022] DELETE /identity/v3/projects/98b7bc38b0aa4939acc4631b85f09047 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:12.975627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df3a7-fa82-4820-a7a6-952a68b25e45 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:12.975928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df3a7-fa82-4820-a7a6-952a68b25e45 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:12.976208 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df3a7-fa82-4820-a7a6-952a68b25e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:12.990996 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d39a69-5ab7-4ef4-95c8-fe62a0f1541a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:12.996287 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d14df3a7-fa82-4820-a7a6-952a68b25e45 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:12.997606 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d14df3a7-fa82-4820-a7a6-952a68b25e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:13.050716 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.051496 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.051811 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.052124 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.052860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.054805 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.065781 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1396/2782] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:13.068370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1387/2783] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:13.070260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.071981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:13.076097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d373cf-4654-4c92-b411-c6044878030d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.077223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d373cf-4654-4c92-b411-c6044878030d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.077640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d373cf-4654-4c92-b411-c6044878030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.086292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0ddd2-11a4-48d6-b2e7-a59727a5fde0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.091153 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.091923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.092234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.092527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] PATH_INFO: `/v3/projects/5f82be918c6d4b5d8067f95d665042cd/users/9d09e356b48940e4a21ec62f978efa3c/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.094588 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f82be918c6d4b5d8067f95d665042cd, user_id=9d09e356b48940e4a21ec62f978efa3c, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.097763 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.099169 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-63d373cf-4654-4c92-b411-c6044878030d None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:13.100399 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-63d373cf-4654-4c92-b411-c6044878030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:13.108749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1388/2784] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:13.108905 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1397/2785] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:13 2022] PUT /identity/v3/projects/5f82be918c6d4b5d8067f95d665042cd/users/9d09e356b48940e4a21ec62f978efa3c/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:13.113862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d373cf-4654-4c92-b411-c6044878030d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.123915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d373cf-4654-4c92-b411-c6044878030d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.188158 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.188892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.189168 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.189646 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.190286 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:13.190722 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.192493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.206901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1389/2786] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:13.211376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.224368 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c230c6-9f54-49a8-a980-69c54666ac5f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.230513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.231418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.231888 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.232371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.233241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.235295 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.267359 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1398/2787] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:13 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:13.272297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.285492 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619bfe8-1092-4129-bb4e-4cf9f948edd4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.334755 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:13.355444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.356048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.356307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.356569 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.357935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.360351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.383652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1390/2788] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:13 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:13.388445 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.403522 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf67ce-15ca-4c27-8003-d7d90bcc7a1b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.410913 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.411880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.412246 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.412611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.413378 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.415158 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.430263 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1399/2789] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:13.434302 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.449011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3b54-47fc-4a09-a6de-89dcec3b086e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.455505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507f58e-155f-464b-b0f5-d1d449618589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.456876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507f58e-155f-464b-b0f5-d1d449618589 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.456876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507f58e-155f-464b-b0f5-d1d449618589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.456876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507f58e-155f-464b-b0f5-d1d449618589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.457749 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507f58e-155f-464b-b0f5-d1d449618589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.459852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507f58e-155f-464b-b0f5-d1d449618589 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.479753 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1391/2790] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:13.489979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.509172 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a657-44e9-4e25-b6ec-20b6578812a3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.515505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.517350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.517673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.518337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] PATH_INFO: `/v3/projects/58c6d82cc44749729759ed0658d4c0b1/users/35f4cb5addfa44a9b0e2fb88e68c7431/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.540347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c6d82cc44749729759ed0658d4c0b1, user_id=35f4cb5addfa44a9b0e2fb88e68c7431, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.546862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.567391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1400/2791] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:13 2022] PUT /identity/v3/projects/58c6d82cc44749729759ed0658d4c0b1/users/35f4cb5addfa44a9b0e2fb88e68c7431/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:13.583945 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.584517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.584834 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.624950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] MFA Rules not processed for user `35f4cb5addfa44a9b0e2fb88e68c7431`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:13.628409 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:13.711603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1392/2792] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:13.713306 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.752960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:13.832350 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e1ca-6856-4e7a-ad1e-2767baadab4b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:13.842722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209ef9-3879-40d3-a1ac-ddb7fd0b05fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:13.843471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209ef9-3879-40d3-a1ac-ddb7fd0b05fb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:13.844570 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209ef9-3879-40d3-a1ac-ddb7fd0b05fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:13.844932 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209ef9-3879-40d3-a1ac-ddb7fd0b05fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:13.845634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b209ef9-3879-40d3-a1ac-ddb7fd0b05fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:13.848600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b209ef9-3879-40d3-a1ac-ddb7fd0b05fb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:13.884271 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1401/2793] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:13 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:13.888627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:13.901295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c5530-e7f1-4676-a637-44bad1d65e55 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.018243 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.019923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.020404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.020713 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.021530 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.023884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.061284 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.061981 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.062146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1393/2794] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:13 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:14.062278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.065803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:14.082803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1767771-d73b-4059-9e05-600c11b7a93f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.088985 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.089620 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.089928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.090174 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.090922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.092769 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.102700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] MFA Rules not processed for user `9d09e356b48940e4a21ec62f978efa3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:14.103967 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:14.107949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1394/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 06:37:14.112459 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:14.125484 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17951e-5020-4996-999e-73a10c46c216 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.135826 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.136436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.136717 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.137106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] PATH_INFO: `/v3/projects/12e0d91e3d78485b8cd5cd4474138740/users/d2bc8a5bad234ef98139b61183d71bcd/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.153341 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e0d91e3d78485b8cd5cd4474138740, user_id=d2bc8a5bad234ef98139b61183d71bcd, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.156536 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.175981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1395/2796] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:14 2022] PUT /identity/v3/projects/12e0d91e3d78485b8cd5cd4474138740/users/d2bc8a5bad234ef98139b61183d71bcd/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:14.177099 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:14.187186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:14.198245 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1402/2797] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:14.212761 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:14.214405 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:14.251934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b05f6-bb5b-4a10-9816-655ee21e54fa admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.257625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.258330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.258688 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.259056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.259925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.262378 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.275577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1396/2798] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:14.280005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:14.291411 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1450b-e22b-44c0-ab63-e3ff7b5d709d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.292937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c4858-9090-4e00-adbd-6ea92e2d6e8b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.297518 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8289d-12b9-47dc-8667-72a23dbcddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.298253 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8289d-12b9-47dc-8667-72a23dbcddb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.298623 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8289d-12b9-47dc-8667-72a23dbcddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.299860 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8289d-12b9-47dc-8667-72a23dbcddb4 admin admin] PATH_INFO: `/v3/projects/12e0d91e3d78485b8cd5cd4474138740/users/d2bc8a5bad234ef98139b61183d71bcd/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.302446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8289d-12b9-47dc-8667-72a23dbcddb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e0d91e3d78485b8cd5cd4474138740, user_id=d2bc8a5bad234ef98139b61183d71bcd, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.305405 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8289d-12b9-47dc-8667-72a23dbcddb4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.319640 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1397/2799] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:14 2022] PUT /identity/v3/projects/12e0d91e3d78485b8cd5cd4474138740/users/d2bc8a5bad234ef98139b61183d71bcd/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:14.329355 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.329641 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.329915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.358758 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.359373 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.359683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.359960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.362430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:14.363782 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] MFA Rules not processed for user `d2bc8a5bad234ef98139b61183d71bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:14.365221 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:14.374600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:14.434680 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.437210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.449212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1403/2800] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:14.458810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1398/2801] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:14.471690 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:14.477123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:14.489648 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7605e-099f-42d4-ad8a-056bc7afd2fa service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:14.562226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:14.562855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:14.563170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:14.563404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:14.634465 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:14.636322 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:14.649644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1404/2802] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:15.892085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:15.902184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebff028-f008-4eab-8a27-3a30208471ff None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:15.906502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:15.907110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:15.907390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:15.907727 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:15.969881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:15.971928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:15.983919 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1399/2803] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:20.474831 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:20.485909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6989d3-5e6d-4fdb-8ba4-9422feea2ec0 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:20.490286 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848d9a6-d554-451c-9463-77de5cdc7acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:20.490988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d9a6-d554-451c-9463-77de5cdc7acb service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:20.491356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d9a6-d554-451c-9463-77de5cdc7acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:20.491702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d9a6-d554-451c-9463-77de5cdc7acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:20.501997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848d9a6-d554-451c-9463-77de5cdc7acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:20.503432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848d9a6-d554-451c-9463-77de5cdc7acb service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:20.513112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1405/2804] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:25.978306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:29.607820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:29.609258 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:29.700636 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb69a1-918c-4f3a-9c08-3253bf154b5c service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:29.705633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:29.706234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:29.706515 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:29.706784 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:29.708524 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:29.720335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:29.722170 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:29.732590 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1400/2805] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:30.508813 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=76 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:30.512739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:30.513225 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:30.514146 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:30.533390 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] MFA Rules not processed for user `d8de3e1c91e343d6924f7f4ce08bf4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:30.534459 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:30.595427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1406/2806] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:36.292946 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:36.299167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:36.317413 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c722e-5d9c-424f-9965-e7277faaf7e5 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:36.325048 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:36.325894 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:36.326890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:36.326890 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:36.328801 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:36.343196 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:36.345014 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:36.359664 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1401/2807] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:36.867283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:36.952211 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230db8c-efda-4620-92dd-b8c0d736077e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:36.956889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:36.957640 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:36.958022 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:36.958400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] PATH_INFO: `/v3/users/ff5d072b1e084b87aadff728b30fb33c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:36.959820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5d072b1e084b87aadff728b30fb33c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:36.961827 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:37.014779 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5d072b1e084b87aadff728b30fb33c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.023347 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5d072b1e084b87aadff728b30fb33c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.023994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5d072b1e084b87aadff728b30fb33c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.024615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5d072b1e084b87aadff728b30fb33c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.035679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5d072b1e084b87aadff728b30fb33c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.057431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1407/2808] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:36 2022] DELETE /identity/v3/users/ff5d072b1e084b87aadff728b30fb33c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:37.225282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:37.238839 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e48fc-d476-4e1f-b163-3cf8f7b1202d service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:37.297800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:37.298436 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:37.298695 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:37.298967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] PATH_INFO: `/v3/projects/010d0fa80bd546668b6d6de9adf8c02c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:37.300100 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] RBAC: Authorizing `identity:delete_project(project_id=010d0fa80bd546668b6d6de9adf8c02c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:37.301980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:37.347875 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010d0fa80bd546668b6d6de9adf8c02c', 'request_id': 'req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.355127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010d0fa80bd546668b6d6de9adf8c02c', 'request_id': 'req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.357295 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:37.362117 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010d0fa80bd546668b6d6de9adf8c02c', 'request_id': 'req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.374461 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] The token cache is being invalidate because project 010d0fa80bd546668b6d6de9adf8c02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:37.374760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.375449 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.376993 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1402/2809] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:37 2022] DELETE /identity/v3/projects/010d0fa80bd546668b6d6de9adf8c02c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:37.380995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:37.382404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:37.399644 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921aed8-e433-490d-8bec-128772fd8d86 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:37.421029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:37.461484 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:37.462127 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:37.462401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:37.462670 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] PATH_INFO: `/v3/users/9d09e356b48940e4a21ec62f978efa3c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:37.463836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d09e356b48940e4a21ec62f978efa3c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:37.465792 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:37.517799 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d09e356b48940e4a21ec62f978efa3c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.525563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d09e356b48940e4a21ec62f978efa3c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.526269 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d09e356b48940e4a21ec62f978efa3c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.526909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d09e356b48940e4a21ec62f978efa3c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.534369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-cfbe8873-2308-4848-86bb-4977ea8871ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d09e356b48940e4a21ec62f978efa3c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.561135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1408/2810] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:37 2022] DELETE /identity/v3/users/9d09e356b48940e4a21ec62f978efa3c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:37.742822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:37.744351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:37.757803 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0c6d2-a0f3-4804-ace0-c4a46fca539b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:37.815207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:37.815870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:37.816151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:37.816424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] PATH_INFO: `/v3/projects/5f82be918c6d4b5d8067f95d665042cd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:37.817532 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f82be918c6d4b5d8067f95d665042cd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:37.819422 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:37.864904 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f82be918c6d4b5d8067f95d665042cd', 'request_id': 'req-4372224f-63e6-4323-8598-ec666b7f1e16'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.872197 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f82be918c6d4b5d8067f95d665042cd', 'request_id': 'req-4372224f-63e6-4323-8598-ec666b7f1e16'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.878350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f82be918c6d4b5d8067f95d665042cd', 'request_id': 'req-4372224f-63e6-4323-8598-ec666b7f1e16'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.890219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] The token cache is being invalidate because project 5f82be918c6d4b5d8067f95d665042cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:37.890510 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.891180 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4372224f-63e6-4323-8598-ec666b7f1e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:37.892720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1403/2811] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:37 2022] DELETE /identity/v3/projects/5f82be918c6d4b5d8067f95d665042cd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:37.897541 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:37.898028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:37.898131 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:37.917949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:37.919073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:37.977738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1409/2812] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:37.985321 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:37.985603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:37.985876 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.006414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:38.007628 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:38.016022 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1404/2813] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:38.020112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.030003 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b717dfe-7df8-4d39-bb3d-3602525654e1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.034532 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.035129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.035408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.035726 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.036369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:38.036673 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.038260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.050743 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1410/2814] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:38.055702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.065602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966153-121e-49eb-be8f-868c6a75dd5c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.070216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.070808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.071085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.071356 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.072012 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.073870 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.100898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1405/2815] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:38 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:38.104732 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.115979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b46cfc-4e98-44c5-b169-dd96b7571bc7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.147478 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:38.178556 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.179076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.179384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.179686 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.180314 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.182274 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.204618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1411/2816] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:38 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:38.208668 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.219126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fc2ec-5c5c-4d95-9eb2-f1cf990f2111 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.224141 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.224820 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.225167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.225506 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.226228 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.227950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.239519 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1406/2817] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:38.243778 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.254951 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794261b9-d1b2-4ca6-ac04-ce20a8373b85 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.259508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908553a1-8b2e-45e8-acf7-4a8bce4e8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.260168 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908553a1-8b2e-45e8-acf7-4a8bce4e8e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.260443 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908553a1-8b2e-45e8-acf7-4a8bce4e8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.260712 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908553a1-8b2e-45e8-acf7-4a8bce4e8e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.261346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908553a1-8b2e-45e8-acf7-4a8bce4e8e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.262915 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908553a1-8b2e-45e8-acf7-4a8bce4e8e0d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.278647 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1412/2818] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:38.281067 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.290835 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad2db3-961c-4713-9892-34dce3489270 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.295933 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.296566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.296841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.297109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] PATH_INFO: `/v3/projects/b3337817756547268c7e905de0c69a32/users/f1b6c74fdd5b42b19e3b624dc79bcff9/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.311836 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3337817756547268c7e905de0c69a32, user_id=f1b6c74fdd5b42b19e3b624dc79bcff9, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.314428 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.327625 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1407/2819] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:38 2022] PUT /identity/v3/projects/b3337817756547268c7e905de0c69a32/users/f1b6c74fdd5b42b19e3b624dc79bcff9/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:38.339365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.339683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.339962 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.368080 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] MFA Rules not processed for user `f1b6c74fdd5b42b19e3b624dc79bcff9`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:38.369292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:38.436490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1413/2820] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:38.447447 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:38.449186 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:38.464351 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:38.508783 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c477-8e78-4b1f-8e3f-694f7425ceae admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:38.512385 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae01359-f5bb-46e3-b7b5-0ece44a5c95b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:38.513009 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae01359-f5bb-46e3-b7b5-0ece44a5c95b service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:38.513307 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae01359-f5bb-46e3-b7b5-0ece44a5c95b service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:38.513577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae01359-f5bb-46e3-b7b5-0ece44a5c95b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:38.521676 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae01359-f5bb-46e3-b7b5-0ece44a5c95b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:38.523112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae01359-f5bb-46e3-b7b5-0ece44a5c95b service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:38.532348 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1408/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:45.636327 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:45.637863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:45.708829 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bb60d-0346-4b99-80fe-6793c747890f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:45.715576 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:45.716409 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:45.716698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:45.717025 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] PATH_INFO: `/v3/users/35f4cb5addfa44a9b0e2fb88e68c7431` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:45.718229 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f4cb5addfa44a9b0e2fb88e68c7431)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:45.720221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:45.763292 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:45.769260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f4cb5addfa44a9b0e2fb88e68c7431'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:45.776890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f4cb5addfa44a9b0e2fb88e68c7431'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:45.777458 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f4cb5addfa44a9b0e2fb88e68c7431'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:45.778047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f4cb5addfa44a9b0e2fb88e68c7431'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:45.786254 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f4cb5addfa44a9b0e2fb88e68c7431'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:45.815048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1414/2822] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:45 2022] DELETE /identity/v3/users/35f4cb5addfa44a9b0e2fb88e68c7431 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:45.819825 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:45.820315 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:45.820509 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:45.839726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:45.840972 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:45.902658 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1409/2823] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:45.913071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:45.914510 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:46.007235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36d1d7-9d5f-45f0-b531-25d4ce22be25 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:46.011489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:46.012123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:46.012397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:46.012689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:46.022343 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:46.023718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:46.034023 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1415/2824] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:46.327455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:46.328934 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:46.342192 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61107a-86cb-4896-9b0a-f8052dca0879 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:46.348652 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:46.349311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:46.349599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:46.349873 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] PATH_INFO: `/v3/projects/58c6d82cc44749729759ed0658d4c0b1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:46.351699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c6d82cc44749729759ed0658d4c0b1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:46.355647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:46.408666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c6d82cc44749729759ed0658d4c0b1', 'request_id': 'req-4f265fde-daf3-4e04-9d01-c488f891762e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.415689 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c6d82cc44749729759ed0658d4c0b1', 'request_id': 'req-4f265fde-daf3-4e04-9d01-c488f891762e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.423106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c6d82cc44749729759ed0658d4c0b1', 'request_id': 'req-4f265fde-daf3-4e04-9d01-c488f891762e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.434092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] The token cache is being invalidate because project 58c6d82cc44749729759ed0658d4c0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:46.434384 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.435881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.437352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1410/2825] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:46 2022] DELETE /identity/v3/projects/58c6d82cc44749729759ed0658d4c0b1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:46.441578 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:46.452222 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc30e2-f1f1-457a-972b-2a80318be9fb service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:46.511495 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:46.512152 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:46.512427 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:46.512699 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] PATH_INFO: `/v3/users/d2bc8a5bad234ef98139b61183d71bcd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:46.513809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2bc8a5bad234ef98139b61183d71bcd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:46.515754 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:46.560676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:46.563881 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2bc8a5bad234ef98139b61183d71bcd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.574422 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2bc8a5bad234ef98139b61183d71bcd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.575455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2bc8a5bad234ef98139b61183d71bcd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.576563 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2bc8a5bad234ef98139b61183d71bcd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.585078 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b49ffc17-2732-4653-be61-a1cece72a4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2bc8a5bad234ef98139b61183d71bcd'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:46.603189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1416/2826] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:46 2022] DELETE /identity/v3/users/d2bc8a5bad234ef98139b61183d71bcd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:46.876720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:46.878184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:46.892414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f265fde-daf3-4e04-9d01-c488f891762e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:46.937811 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:46.958279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:46.958916 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:46.959200 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:46.959479 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] PATH_INFO: `/v3/projects/12e0d91e3d78485b8cd5cd4474138740` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:46.960630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e0d91e3d78485b8cd5cd4474138740)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:46.962534 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.011256 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e0d91e3d78485b8cd5cd4474138740', 'request_id': 'req-3f82b1c1-61ed-4c63-bd7a-5876309649eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:47.019346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e0d91e3d78485b8cd5cd4474138740', 'request_id': 'req-3f82b1c1-61ed-4c63-bd7a-5876309649eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:47.025093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e0d91e3d78485b8cd5cd4474138740', 'request_id': 'req-3f82b1c1-61ed-4c63-bd7a-5876309649eb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:47.038743 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] The token cache is being invalidate because project 12e0d91e3d78485b8cd5cd4474138740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:47.039030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:47.039744 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3f82b1c1-61ed-4c63-bd7a-5876309649eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:47.041233 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1411/2827] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:46 2022] DELETE /identity/v3/projects/12e0d91e3d78485b8cd5cd4474138740 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:47.048279 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.048821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.048821 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.076019 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:47.077167 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:47.136202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1417/2828] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:47.145981 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.146265 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.146538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.167037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:47.168260 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:47.176410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1412/2829] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:47.180684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.190304 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9790335-e7e6-42db-9d01-62c107b2a641 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.194912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.195515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.195824 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.196092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.196734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:47.197037 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.198617 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.212746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1418/2830] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 06:37:47.217828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.228937 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbef35-94fd-48b6-a51a-4a022bb9fffc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.233748 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.234372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.234653 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.234925 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.235633 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.237563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.265473 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1413/2831] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:47 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:47.269816 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.280497 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758e400-e089-41ba-af88-2d936c847911 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.345985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.347027 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.347310 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.347685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.348476 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.350511 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.374722 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1419/2832] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:47 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:47.376007 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.386431 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319dfff-ec6b-4570-997f-adc106700657 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.391841 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.392278 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.392800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.393083 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.393930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.395573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.406663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1414/2833] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:47.410738 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.421642 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dee82-832a-45b6-b7d4-f2da106bfa7a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.426381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ac40-cf40-4701-876e-29a17c71ddf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.426979 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ac40-cf40-4701-876e-29a17c71ddf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.427260 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ac40-cf40-4701-876e-29a17c71ddf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.427585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ac40-cf40-4701-876e-29a17c71ddf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.428227 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ac40-cf40-4701-876e-29a17c71ddf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.429803 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ac40-cf40-4701-876e-29a17c71ddf9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.441914 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1420/2834] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:47.446311 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.457078 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8e4d0-2569-48db-b765-f686fa89586e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.461720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.462370 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.462650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.462922 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] PATH_INFO: `/v3/projects/428599309193462db9d00e67bbee51cc/users/b7a7598585184f97acfb3dec6825c432/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.477576 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428599309193462db9d00e67bbee51cc, user_id=b7a7598585184f97acfb3dec6825c432, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.480262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.493577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1415/2835] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:47 2022] PUT /identity/v3/projects/428599309193462db9d00e67bbee51cc/users/b7a7598585184f97acfb3dec6825c432/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:47.505112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57942-fec7-4859-9405-e70413fd407d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.505363 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57942-fec7-4859-9405-e70413fd407d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.505624 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57942-fec7-4859-9405-e70413fd407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.535789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-35f57942-fec7-4859-9405-e70413fd407d None None] MFA Rules not processed for user `b7a7598585184f97acfb3dec6825c432`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:47.537058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-35f57942-fec7-4859-9405-e70413fd407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:47.556183 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:47.620592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1421/2836] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:47.631376 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:47.633214 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:47.722112 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:47.726477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6780f2e-39da-48d6-b5a4-466d193839e1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:47.732322 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e51bb3-7587-4481-8318-e1ef4042153b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:47.733525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e51bb3-7587-4481-8318-e1ef4042153b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:47.733991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e51bb3-7587-4481-8318-e1ef4042153b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:47.734433 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e51bb3-7587-4481-8318-e1ef4042153b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:47.743424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e51bb3-7587-4481-8318-e1ef4042153b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:47.744891 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e51bb3-7587-4481-8318-e1ef4042153b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:47.754750 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1416/2837] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:50.868957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57942-fec7-4859-9405-e70413fd407d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:50.932782 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57942-fec7-4859-9405-e70413fd407d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:50.937376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:50.938007 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:50.938283 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:50.938555 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] PATH_INFO: `/v3/users/b7a7598585184f97acfb3dec6825c432` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:50.939715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a7598585184f97acfb3dec6825c432)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:50.941730 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:50.992453 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a7598585184f97acfb3dec6825c432'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:50.999571 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a7598585184f97acfb3dec6825c432'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.000238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a7598585184f97acfb3dec6825c432'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.000875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a7598585184f97acfb3dec6825c432'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.007889 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a7598585184f97acfb3dec6825c432'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.025760 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1422/2838] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:50 2022] DELETE /identity/v3/users/b7a7598585184f97acfb3dec6825c432 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:51.031074 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.031575 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.031773 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.050346 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:51.051674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:51.113808 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1417/2839] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:51.123677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.125112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:51.167431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:51.186446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631145ae-eb1f-4a9d-bda7-a2309977b1b0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.190076 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cb48c-2d83-4b9f-97f6-40fca15ec423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.190679 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cb48c-2d83-4b9f-97f6-40fca15ec423 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.190959 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cb48c-2d83-4b9f-97f6-40fca15ec423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.191233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cb48c-2d83-4b9f-97f6-40fca15ec423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.200643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cb48c-2d83-4b9f-97f6-40fca15ec423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.202233 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cb48c-2d83-4b9f-97f6-40fca15ec423 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.212123 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1423/2840] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:51.483131 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.484605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:51.501220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4471-f424-4dbe-89bd-d5b4a4356cea None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.506543 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.507188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.507529 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.507738 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] PATH_INFO: `/v3/projects/428599309193462db9d00e67bbee51cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.508929 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=428599309193462db9d00e67bbee51cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.510845 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.559189 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428599309193462db9d00e67bbee51cc', 'request_id': 'req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.567444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428599309193462db9d00e67bbee51cc', 'request_id': 'req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.573915 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428599309193462db9d00e67bbee51cc', 'request_id': 'req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.585143 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] The token cache is being invalidate because project 428599309193462db9d00e67bbee51cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:51.585424 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.586057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fff00f92-7e7c-4f10-a93c-e876ec1ac54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:51.587504 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1418/2841] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:51 2022] DELETE /identity/v3/projects/428599309193462db9d00e67bbee51cc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:51.592742 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.593058 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.593330 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.614498 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:51.615750 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:51.676548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1424/2842] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:51.684055 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.684432 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.685335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.704960 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:51.706175 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:51.713954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1419/2843] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:51.718251 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.728144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d224e-3ff0-40bf-9bff-b8950728d753 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.733302 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.733893 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.734178 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.734446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.735086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:51.735386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.737001 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.748685 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1425/2844] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:51.754294 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.764266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52954-0bbc-4fd7-ac31-40e64da8eaf1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.768909 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.769505 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.769787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.770085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.770726 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.772657 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.800438 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1420/2845] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:51 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:51.804592 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.814604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d858de7-e501-4737-b294-a579684c1f9b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.870339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.870963 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.871199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.871481 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.873285 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.874904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.896785 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1426/2846] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:51 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:51.900799 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.911581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52694a52-861b-43c4-b3c0-c606c7427638 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.916096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.916700 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.916980 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.917254 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.917900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.919822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.929942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1421/2847] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:51.933855 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.944385 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4432840-6028-4e4e-a7c9-f83cc436aef2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.949057 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f577286-3afb-424d-a98f-d3515311c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.949652 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f577286-3afb-424d-a98f-d3515311c4dc admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.950585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f577286-3afb-424d-a98f-d3515311c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.950746 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f577286-3afb-424d-a98f-d3515311c4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.951515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f577286-3afb-424d-a98f-d3515311c4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:51.953162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f577286-3afb-424d-a98f-d3515311c4dc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:51.963641 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1427/2848] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:51.967644 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:51.977797 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb4d51-81ff-4979-8d19-6a552bc3a3b0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:51.982647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:51.983310 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:51.983642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:51.983924 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] PATH_INFO: `/v3/projects/48ff0c86cc8a4709842caf56df67f458/users/1fdf6f38b6ac4321a3a28075e43eff18/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:51.998702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ff0c86cc8a4709842caf56df67f458, user_id=1fdf6f38b6ac4321a3a28075e43eff18, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:52.001361 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:52.014712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1422/2849] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:51 2022] PUT /identity/v3/projects/48ff0c86cc8a4709842caf56df67f458/users/1fdf6f38b6ac4321a3a28075e43eff18/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:52.024764 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:52.025047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:52.025315 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:52.054683 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] MFA Rules not processed for user `1fdf6f38b6ac4321a3a28075e43eff18`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:52.056381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:52.098584 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:52.131850 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1428/2850] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:52.143693 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:52.145421 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:52.156262 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:52.209550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0461af-62cb-47ea-a5ff-e612a8455d58 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:52.213006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1c1f4-2bfa-463f-a4c7-48eaf30c0224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f236664fa09b4a4aafe6980768fc263b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:52.213602 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1c1f4-2bfa-463f-a4c7-48eaf30c0224 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:52.213878 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1c1f4-2bfa-463f-a4c7-48eaf30c0224 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:52.214146 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1c1f4-2bfa-463f-a4c7-48eaf30c0224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:52.222910 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1c1f4-2bfa-463f-a4c7-48eaf30c0224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:52.224544 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1c1f4-2bfa-463f-a4c7-48eaf30c0224 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:52.233337 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1423/2851] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:52.259610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:52.329189 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d61e93-d89d-4a58-aaf7-2e32bd2478c4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:52.333724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:52.334348 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:52.334622 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:52.334891 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] PATH_INFO: `/v3/users/1fdf6f38b6ac4321a3a28075e43eff18` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:52.336028 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdf6f38b6ac4321a3a28075e43eff18)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:52.337931 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:52.384957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdf6f38b6ac4321a3a28075e43eff18'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:52.392787 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdf6f38b6ac4321a3a28075e43eff18'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:52.393391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdf6f38b6ac4321a3a28075e43eff18'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:52.393983 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdf6f38b6ac4321a3a28075e43eff18'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:52.401094 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdf6f38b6ac4321a3a28075e43eff18'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:52.418325 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1429/2852] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:37:52 2022] DELETE /identity/v3/users/1fdf6f38b6ac4321a3a28075e43eff18 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:52.423271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:52.423581 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:52.423865 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:52.446210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:52.447423 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:52.516445 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1424/2853] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:52.526171 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:52.527657 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:52.605492 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd779a19-80cb-4bee-9f0b-fb8111d9b2eb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:52.609952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebb032-e3e5-4eb4-834d-dc6b63af261d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:52.610569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebb032-e3e5-4eb4-834d-dc6b63af261d service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:52.610847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebb032-e3e5-4eb4-834d-dc6b63af261d service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:52.613919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebb032-e3e5-4eb4-834d-dc6b63af261d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:52.619164 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:52.621986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebb032-e3e5-4eb4-834d-dc6b63af261d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:52.624085 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebb032-e3e5-4eb4-834d-dc6b63af261d service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:52.639370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1430/2854] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:52.960866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:52.962387 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:52.981225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0872d-c9e3-4390-8386-fa6cdacedb25 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:52.986905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:52.987455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:52.987787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:52.988056 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] PATH_INFO: `/v3/projects/48ff0c86cc8a4709842caf56df67f458` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:52.989166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ff0c86cc8a4709842caf56df67f458)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:52.991053 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.044414 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ff0c86cc8a4709842caf56df67f458', 'request_id': 'req-1f790735-3a14-485b-a709-3902891a13dc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:53.052093 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ff0c86cc8a4709842caf56df67f458', 'request_id': 'req-1f790735-3a14-485b-a709-3902891a13dc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:53.059634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ff0c86cc8a4709842caf56df67f458', 'request_id': 'req-1f790735-3a14-485b-a709-3902891a13dc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:53.071328 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] The token cache is being invalidate because project 48ff0c86cc8a4709842caf56df67f458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:37:53.071696 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:53.072332 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1f790735-3a14-485b-a709-3902891a13dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:37:53.073809 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1425/2855] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:37:52 2022] DELETE /identity/v3/projects/48ff0c86cc8a4709842caf56df67f458 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:53.079583 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.080071 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.080202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.100946 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:53.102141 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:53.163512 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1431/2856] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:53.171733 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.172266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.172266 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.192903 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:53.194091 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:53.202770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1426/2857] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:53.206958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.217523 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4af7a-78dd-4d33-a6b7-51c47aa3dac9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.222457 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.223055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.223339 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.223656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.224360 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:37:53.224665 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.226257 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.237912 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1432/2858] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:37:53.242219 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.251525 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedd01d-6e66-4257-8df3-6197c2b21dc0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.256135 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.256728 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.257005 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.257275 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.257899 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.259818 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.288194 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1427/2859] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:53 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:53.292238 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.302558 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c398b0-dfa6-4de8-af85-3e57ba9909a8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.359210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.359836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.360113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.360387 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.361013 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.362908 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.382431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1433/2860] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:53 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:53.386498 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.397271 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ccc16d-f436-4145-a089-986bbfa1152e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.401914 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.402502 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.402778 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.403096 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.404071 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.405632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.407921 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:53.418493 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1428/2861] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:53.422446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.432337 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ab7e6-eb10-4a99-8e95-a0547c7cf182 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.437424 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164dd81-b0f4-49c7-bff0-1b01bc3cb397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.438018 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164dd81-b0f4-49c7-bff0-1b01bc3cb397 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.438294 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164dd81-b0f4-49c7-bff0-1b01bc3cb397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.438566 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164dd81-b0f4-49c7-bff0-1b01bc3cb397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.439232 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164dd81-b0f4-49c7-bff0-1b01bc3cb397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.440926 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164dd81-b0f4-49c7-bff0-1b01bc3cb397 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.453089 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1434/2862] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:53.457101 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.467187 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58553250-a418-42d3-bf1c-b243dc930ae0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.472080 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.472711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.472988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.473257 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] PATH_INFO: `/v3/projects/3312523776514be3a56e169c9234168b/users/baef9538b75b4244b6ccf0922af7e5b1/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.489072 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3312523776514be3a56e169c9234168b, user_id=baef9538b75b4244b6ccf0922af7e5b1, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.491796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.505868 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1429/2863] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:53 2022] PUT /identity/v3/projects/3312523776514be3a56e169c9234168b/users/baef9538b75b4244b6ccf0922af7e5b1/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:37:53.510906 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c1523-b316-463a-8a9e-bb36dfd8f09a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.511195 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c1523-b316-463a-8a9e-bb36dfd8f09a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.511471 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c1523-b316-463a-8a9e-bb36dfd8f09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.530723 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-121c1523-b316-463a-8a9e-bb36dfd8f09a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:53.531970 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-121c1523-b316-463a-8a9e-bb36dfd8f09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:53.591430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1435/2864] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:53.601392 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:53.603075 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:53.659950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b070f48-d08c-4477-bbe2-94734f4d7e42 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:53.663523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:53.664154 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:53.664446 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:53.664702 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:53.673656 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:53.675109 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:53.685250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1430/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:37:55.214859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd10d04-8cf3-4e85-a400-ca8e4bcbedeb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:55.216377 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd10d04-8cf3-4e85-a400-ca8e4bcbedeb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:55.216842 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd10d04-8cf3-4e85-a400-ca8e4bcbedeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:55.224391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:55.260077 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dcd10d04-8cf3-4e85-a400-ca8e4bcbedeb None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:55.265846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dcd10d04-8cf3-4e85-a400-ca8e4bcbedeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:55.334744 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1436/2866] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:37:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:37:55.386017 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:55.387605 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:55.466234 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117be877-94a1-4c30-bdea-afb0a8dcba67 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:55.471470 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:55.472127 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:55.472408 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:55.472678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:55.481730 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:55.483299 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:55.491887 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1431/2867] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:37:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:37:58.579589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:58.579589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:58.579589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:58.618159 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] MFA Rules not processed for user `baef9538b75b4244b6ccf0922af7e5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:37:58.619401 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:37:58.719689 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1437/2868] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:37:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:37:58.725371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:58.739538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7d2d0-986b-4392-961c-e82fb307e31c service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:58.746191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:58.748670 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:58.749591 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:58.750092 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:58.751103 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:58.754930 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:58.809624 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1432/2869] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:37:58 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:37:58.814250 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:58.829500 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447654-e37d-4f54-9217-aa1c2ff8357a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:58.922219 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:58.923075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:58.923411 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:58.923851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:58.924667 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:58.927408 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:58.958591 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1438/2870] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:37:58 2022] POST /identity/v3/users => generated 298 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 06:37:58.962711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:58.976021 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470db58-b184-4186-b46e-eaf7dc5fa468 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:58.980953 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:58.982616 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:58.983032 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:58.983404 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:58.984252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:58.988140 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:59.000800 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1433/2871] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:59.004994 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:59.020975 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f04dc-9745-4b6f-a6d8-12502925981e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:59.030859 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:59.031773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:59.032143 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:59.032531 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] PATH_INFO: `/v3/projects/26d791d43e96429dbba858b56dbf73f2/users/200e1ff9d8c946b288d1e4b69640ee64/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:59.048460 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:59.062613 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d791d43e96429dbba858b56dbf73f2, user_id=200e1ff9d8c946b288d1e4b69640ee64, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:59.065261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:59.082012 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1439/2872] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:59 2022] PUT /identity/v3/projects/26d791d43e96429dbba858b56dbf73f2/users/200e1ff9d8c946b288d1e4b69640ee64/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:37:59.085882 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:59.098478 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bea3d2-ea20-47ea-9393-119183169f91 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:59.156828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:37:59.188437 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:59.189188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:59.189511 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:59.189872 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:59.190603 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:59.192331 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:59.207732 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1434/2873] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:37:59.208676 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:37:59.221695 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d22862-03f9-4568-877c-82a93e8ea347 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:37:59.226949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07012-4b58-4e4e-a089-a822599f9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:37:59.227653 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07012-4b58-4e4e-a089-a822599f9e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:37:59.227937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07012-4b58-4e4e-a089-a822599f9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:37:59.228234 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07012-4b58-4e4e-a089-a822599f9e8b admin admin] PATH_INFO: `/v3/projects/26d791d43e96429dbba858b56dbf73f2/users/200e1ff9d8c946b288d1e4b69640ee64/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:37:59.230137 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07012-4b58-4e4e-a089-a822599f9e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d791d43e96429dbba858b56dbf73f2, user_id=200e1ff9d8c946b288d1e4b69640ee64, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:37:59.233862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07012-4b58-4e4e-a089-a822599f9e8b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:37:59.247242 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1440/2874] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:37:59 2022] PUT /identity/v3/projects/26d791d43e96429dbba858b56dbf73f2/users/200e1ff9d8c946b288d1e4b69640ee64/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:00.924097 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:00.925827 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:00.945466 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e1b3f-26e1-4ed4-8f61-65b0871a54e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:01.074852 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:01.075990 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:01.076383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:01.076795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] PATH_INFO: `/v3/users/f1b6c74fdd5b42b19e3b624dc79bcff9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:01.078335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b6c74fdd5b42b19e3b624dc79bcff9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:01.082520 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:01.170015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b6c74fdd5b42b19e3b624dc79bcff9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:01.186819 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b6c74fdd5b42b19e3b624dc79bcff9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:01.187680 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b6c74fdd5b42b19e3b624dc79bcff9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:01.206671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b6c74fdd5b42b19e3b624dc79bcff9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:01.239964 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b6c74fdd5b42b19e3b624dc79bcff9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:01.255383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1435/2875] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:38:00 2022] DELETE /identity/v3/users/f1b6c74fdd5b42b19e3b624dc79bcff9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:01.256837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:01.257724 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:01.258223 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:01.285126 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:01.287490 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:01.431977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1441/2876] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:38:01.452185 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:01.453706 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:01.596546 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:01.610795 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9530c7-1c12-4232-a911-4101ef1f5eeb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:01.616627 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995d117-2f67-47d1-9616-8abca7876788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:01.616939 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995d117-2f67-47d1-9616-8abca7876788 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:01.617216 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995d117-2f67-47d1-9616-8abca7876788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:01.617514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995d117-2f67-47d1-9616-8abca7876788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:01.640150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995d117-2f67-47d1-9616-8abca7876788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:01.641235 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995d117-2f67-47d1-9616-8abca7876788 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:01.653297 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1436/2877] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:38:02.535313 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:02.537369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:02.568391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbac72-cc82-48f9-a2ec-82f4256cafdc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:02.578784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:02.580950 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:02.581433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:02.581917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] PATH_INFO: `/v3/projects/b3337817756547268c7e905de0c69a32` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:02.583455 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3337817756547268c7e905de0c69a32)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:02.585639 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:02.685698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3337817756547268c7e905de0c69a32', 'request_id': 'req-c8e69c03-9549-4f11-a46b-23d8c2da4075'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:02.692875 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3337817756547268c7e905de0c69a32', 'request_id': 'req-c8e69c03-9549-4f11-a46b-23d8c2da4075'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:02.715907 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3337817756547268c7e905de0c69a32', 'request_id': 'req-c8e69c03-9549-4f11-a46b-23d8c2da4075'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:02.748737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] The token cache is being invalidate because project b3337817756547268c7e905de0c69a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:38:02.748737 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:02.749960 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c8e69c03-9549-4f11-a46b-23d8c2da4075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:02.752489 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1442/2878] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:38:02 2022] DELETE /identity/v3/projects/b3337817756547268c7e905de0c69a32 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:02.756949 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:02.757455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:02.757592 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:02.806948 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:02.808165 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:03.020898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1437/2879] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:03.031045 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.031847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.032308 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.080909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:03.082678 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:03.104809 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1443/2880] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:03.109604 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:03.129401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e00b9-f932-4592-a717-b54cee1d3365 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:03.147042 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:03.147715 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.148115 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.148391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.149217 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 06:38:03.149531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:03.151220 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:03.181796 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1438/2881] 10.222.0.25 () {58 vars in 1153 bytes} [Fri Sep 9 06:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 06:38:03.187046 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:03.205053 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ebee0-6efe-40f4-ac53-c5bf8b05ccff None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:03.223916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:03.223916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.223916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.223916 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.224634 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:03.232818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:03.296336 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1444/2882] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:38:03 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:38:03.298381 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:03.317227 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46bb4e-0707-4d82-ae51-eaddad3216db admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:03.459126 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:03.461938 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.462236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.462514 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.463195 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:03.469374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:03.537128 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1439/2883] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:38:03 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:38:03.541194 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:03.558572 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3648e0-01cf-46a2-83b4-15d442637130 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:03.581447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:03.585577 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.586206 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.587971 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.590464 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:03.612113 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:03.638404 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1445/2884] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:38:03.640495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:03.655647 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471aad9e-c4ea-4e1f-b30e-6df394f75c8a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:03.663033 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b77fe-4289-48ac-ad63-8519aca9accb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:03.664031 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b77fe-4289-48ac-ad63-8519aca9accb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.664927 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b77fe-4289-48ac-ad63-8519aca9accb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.665372 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b77fe-4289-48ac-ad63-8519aca9accb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.666855 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b77fe-4289-48ac-ad63-8519aca9accb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:03.670150 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b77fe-4289-48ac-ad63-8519aca9accb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:03.712390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1440/2885] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:38:03.713575 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:03.733214 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400cd9c-b1e9-4ded-ac47-49560dae9cf0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:03.755391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:03.756179 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.756520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.756866 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] PATH_INFO: `/v3/projects/240d1fbee4bc4dbeb359f463302be8ee/users/b84d34323691437b9e2e9e99c29db0f1/roles/660e237685d342e8bb6bef00916bca0b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.825846 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240d1fbee4bc4dbeb359f463302be8ee, user_id=b84d34323691437b9e2e9e99c29db0f1, role_id=660e237685d342e8bb6bef00916bca0b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:03.828713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:03.831212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:03.860092 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1446/2886] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:38:03 2022] PUT /identity/v3/projects/240d1fbee4bc4dbeb359f463302be8ee/users/b84d34323691437b9e2e9e99c29db0f1/roles/660e237685d342e8bb6bef00916bca0b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:03.861807 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8754a3-aeeb-4189-afe5-2f3a160d399a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:03.862282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8754a3-aeeb-4189-afe5-2f3a160d399a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:03.862550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8754a3-aeeb-4189-afe5-2f3a160d399a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:03.906550 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5a8754a3-aeeb-4189-afe5-2f3a160d399a None None] MFA Rules not processed for user `285daab328104f75886dde4ae3928fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:03.910823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5a8754a3-aeeb-4189-afe5-2f3a160d399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:04.054928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:04.088570 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1441/2887] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:04.099319 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:04.101603 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:04.248356 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27902f93-a4af-4ec7-a4df-2d8e4ac72ef1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:04.256154 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:04.256948 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:04.257162 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:04.257432 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:04.288259 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:04.289797 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:04.325505 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1447/2888] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:38:06.429102 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708939f-d579-4ab3-b1e5-34a06ed7364c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:06.430046 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708939f-d579-4ab3-b1e5-34a06ed7364c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:06.430538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708939f-d579-4ab3-b1e5-34a06ed7364c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:06.495666 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6708939f-d579-4ab3-b1e5-34a06ed7364c None None] MFA Rules not processed for user `200e1ff9d8c946b288d1e4b69640ee64`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:06.497751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6708939f-d579-4ab3-b1e5-34a06ed7364c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:06.685335 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1442/2889] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:06.703381 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:06.705527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:06.839388 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf9c0-3a7a-4f3a-9d35-e9e15a9c8bd9 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:06.844352 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8bf35f310b4b558dde597cde49655f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:06.845804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:06.846527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:06.846978 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:06.848957 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:06.941752 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:06.943618 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:06.957376 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1448/2890] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:38:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:38:07.267209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aed7ff-fa56-45b7-89cd-12df96d3e57a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:07.267900 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aed7ff-fa56-45b7-89cd-12df96d3e57a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:07.268396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aed7ff-fa56-45b7-89cd-12df96d3e57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:07.322281 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-75aed7ff-fa56-45b7-89cd-12df96d3e57a None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:07.323905 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-75aed7ff-fa56-45b7-89cd-12df96d3e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:07.448225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1443/2891] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:38:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:07.472221 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:07.474418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:07.624093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:07.671061 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cf76b-28eb-4ebe-94e7-2e746414eafe service glance] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:07.678488 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:07.679482 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:07.680544 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:07.681010 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:07.694925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:07.701611 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:07.714469 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1449/2892] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:38:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:38:12.773110 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:12.773573 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:12.773867 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:12.817138 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] MFA Rules not processed for user `b84d34323691437b9e2e9e99c29db0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:12.818612 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:12.923343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1444/2893] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:12.927952 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:12.940601 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203cdb9-a8da-4e80-9686-da29c6130e56 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:12.946264 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:12.946997 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:12.947346 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:12.947721 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:12.948433 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:12.951112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:12.995742 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:13.008984 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1450/2894] 10.222.0.25 () {60 vars in 1153 bytes} [Fri Sep 9 06:38:12 2022] POST /identity/v3/projects => generated 369 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:38:13.041037 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72cd84-74d9-460d-a038-44b447196bd4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:13.065599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:13.141941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:13.142683 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:13.143036 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:13.143382 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:13.144252 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:13.146396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:13.188950 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1445/2895] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Sep 9 06:38:12 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 06:38:13.192890 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:13.209632 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51d0f5-abde-42ff-9742-c6ede8f67040 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:13.215698 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:13.217268 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:13.217604 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:13.217965 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:13.218751 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:13.220747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:13.238739 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1451/2896] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:38:13.243663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:13.255043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6778dbb-4798-41d6-b760-717d946a0652 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:13.264954 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:13.265764 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:13.266043 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:13.266427 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] PATH_INFO: `/v3/projects/e2a72949ee104e32b5cf777e7fe3de8e/users/5c1db29862c04c4d812ecb467d058992/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:13.292004 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a72949ee104e32b5cf777e7fe3de8e, user_id=5c1db29862c04c4d812ecb467d058992, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:13.296495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:13.319832 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1446/2897] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:38:13 2022] PUT /identity/v3/projects/e2a72949ee104e32b5cf777e7fe3de8e/users/5c1db29862c04c4d812ecb467d058992/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:13.323826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:13.334756 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa3d55-a5f4-4ef2-a305-6eb037f56937 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:13.423958 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e05ab-419d-4389-a736-7faff461397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:13.424612 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e05ab-419d-4389-a736-7faff461397d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:13.424861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e05ab-419d-4389-a736-7faff461397d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:13.425185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e05ab-419d-4389-a736-7faff461397d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:13.425991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e05ab-419d-4389-a736-7faff461397d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:13.430323 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e05ab-419d-4389-a736-7faff461397d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:13.472420 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1452/2898] 10.222.0.25 () {58 vars in 1122 bytes} [Fri Sep 9 06:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 06:38:13.476323 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:13.508206 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288b259-4e07-431c-b8b3-91175552612c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:13.521368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:13.522077 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:13.522357 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:13.522634 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] PATH_INFO: `/v3/projects/e2a72949ee104e32b5cf777e7fe3de8e/users/5c1db29862c04c4d812ecb467d058992/roles/835f58ed7a1644f6975d2ad388d5b638` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:13.528455 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a72949ee104e32b5cf777e7fe3de8e, user_id=5c1db29862c04c4d812ecb467d058992, role_id=835f58ed7a1644f6975d2ad388d5b638)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:13.536151 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:13.552941 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1447/2899] 10.222.0.25 () {60 vars in 1483 bytes} [Fri Sep 9 06:38:13 2022] PUT /identity/v3/projects/e2a72949ee104e32b5cf777e7fe3de8e/users/5c1db29862c04c4d812ecb467d058992/roles/835f58ed7a1644f6975d2ad388d5b638 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:22.196520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f34b-ae64-4ecc-9671-6388e2d961d4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:22.197093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f34b-ae64-4ecc-9671-6388e2d961d4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:22.197093 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f34b-ae64-4ecc-9671-6388e2d961d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:22.252446 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-22e8f34b-ae64-4ecc-9671-6388e2d961d4 None None] MFA Rules not processed for user `5c1db29862c04c4d812ecb467d058992`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:22.254684 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22e8f34b-ae64-4ecc-9671-6388e2d961d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:22.431212 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1453/2900] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:22.454637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:22.456632 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:22.753288 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909a303-503f-4ce8-a0a5-2a91b83c0986 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:22.759630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fd97a-aff6-4e72-b4d6-33ec9d1d3f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:22.760350 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fd97a-aff6-4e72-b4d6-33ec9d1d3f65 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:22.760637 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fd97a-aff6-4e72-b4d6-33ec9d1d3f65 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:22.760918 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fd97a-aff6-4e72-b4d6-33ec9d1d3f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:22.870319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:22.903030 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fd97a-aff6-4e72-b4d6-33ec9d1d3f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:22.905054 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fd97a-aff6-4e72-b4d6-33ec9d1d3f65 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:22.916745 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1448/2901] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:38:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:38:25.206582 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c229b-2e9b-4299-bdb9-255c25f84db0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:25.207902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c229b-2e9b-4299-bdb9-255c25f84db0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:25.208184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c229b-2e9b-4299-bdb9-255c25f84db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:25.237704 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a57c229b-2e9b-4299-bdb9-255c25f84db0 None None] MFA Rules not processed for user `b84d34323691437b9e2e9e99c29db0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:25.239184 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a57c229b-2e9b-4299-bdb9-255c25f84db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:25.251307 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1454/2902] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 06:38:26.336480 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:26.337232 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:26.337566 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:26.357223 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] MFA Rules not processed for user `baef9538b75b4244b6ccf0922af7e5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:26.358410 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:26.416444 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1449/2903] 10.222.0.25 () {58 vars in 958 bytes} [Fri Sep 9 06:38:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:26.847361 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d0137-4421-4f90-a2f3-8252faa0b80c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:26.847849 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d0137-4421-4f90-a2f3-8252faa0b80c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:26.848042 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d0137-4421-4f90-a2f3-8252faa0b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:26.874111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c44d0137-4421-4f90-a2f3-8252faa0b80c None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:26.875354 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c44d0137-4421-4f90-a2f3-8252faa0b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:26.895974 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:26.945700 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1455/2904] 10.222.0.25 () {58 vars in 967 bytes} [Fri Sep 9 06:38:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:26.958671 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:27.050061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e079d-3a9b-4cb3-ac47-127d5c39e63c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:27.057563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:27.058596 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:27.058969 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:27.059390 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:27.084354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:27.086371 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:27.098383 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1450/2905] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:38:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:38:28.628144 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:28.628677 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:28.628925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:28.650506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:38:28.658226 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:28.667096 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1456/2906] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Sep 9 06:38:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:38:28.681182 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:28.694483 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582d2e1-330b-4d38-bee3-ea6b1a590ba3 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:28.705367 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:28.706011 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:28.706296 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:28.706577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:28.720622 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:28.723396 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:28.755785 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1451/2907] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:38:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:38:29.763879 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:29.766016 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:30.017917 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f83577-cd82-4b7a-8d93-c48cf55614e5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:30.023527 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:30.024447 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:30.024729 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:30.025115 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] PATH_INFO: `/v3/users/d8de3e1c91e343d6924f7f4ce08bf4e0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:30.037818 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8de3e1c91e343d6924f7f4ce08bf4e0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:30.040923 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:30.182629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8de3e1c91e343d6924f7f4ce08bf4e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:30.228267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8de3e1c91e343d6924f7f4ce08bf4e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:30.229110 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8de3e1c91e343d6924f7f4ce08bf4e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:30.231865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8de3e1c91e343d6924f7f4ce08bf4e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:30.244847 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8de3e1c91e343d6924f7f4ce08bf4e0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:30.305886 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1457/2908] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:38:29 2022] DELETE /identity/v3/users/d8de3e1c91e343d6924f7f4ce08bf4e0 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:30.817352 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:30.841823 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72022995-5c8f-469c-96f2-fc58436da514 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:30.978991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:30.978991 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:30.979712 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:30.980210 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] PATH_INFO: `/v3/projects/b5ecb76de8c443ba93d919ab2bad03c2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:30.987935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ecb76de8c443ba93d919ab2bad03c2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:30.989984 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:31.005912 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:31.139942 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ecb76de8c443ba93d919ab2bad03c2', 'request_id': 'req-73da0d9c-9562-450b-9350-ef4fc8e403c5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:31.201360 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ecb76de8c443ba93d919ab2bad03c2', 'request_id': 'req-73da0d9c-9562-450b-9350-ef4fc8e403c5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:31.210306 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ecb76de8c443ba93d919ab2bad03c2', 'request_id': 'req-73da0d9c-9562-450b-9350-ef4fc8e403c5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:31.226426 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] The token cache is being invalidate because project b5ecb76de8c443ba93d919ab2bad03c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:38:31.227222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:31.228222 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:38:31.230238 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1452/2909] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:38:30 2022] DELETE /identity/v3/projects/b5ecb76de8c443ba93d919ab2bad03c2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:38:40.284278 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:41.222538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:38:55.559767 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:38:55.561210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:55.622358 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeccbe50-db8e-4d01-8eea-e5c1cee6461d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:38:55.671537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:38:55.672170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:38:55.672450 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:38:55.672725 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:38:55.674628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:38:55.738508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:38:55.739995 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:38:55.749973 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1458/2910] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:38:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:39:02.413830 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:02.415236 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:02.427279 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da0d9c-9562-450b-9350-ef4fc8e403c5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:02.430751 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:02.431339 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:02.431862 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:02.434157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:02.435833 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:02.499320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:02.500757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:02.510119 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1453/2911] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:39:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:39:05.745470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=70 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:39:12.505881 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=30 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:39:13.025865 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:13.034715 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b60ec7-d31a-4a13-a9d2-cd20a2ebc7c1 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:13.038329 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:13.038925 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:13.039207 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:13.039485 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:13.041267 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:13.105949 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:13.107454 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:13.117236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1459/2912] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:39:19.087448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:19.089810 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:19.112495 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391d92d-8f6c-4d72-9434-c187c5fdbfa4 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:19.119477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:19.121160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:19.121607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:19.122025 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] PATH_INFO: `/v3/users/472b6623f57a41568fe7ffce8e40e350` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:19.124057 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=472b6623f57a41568fe7ffce8e40e350)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:19.127523 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:19.189898 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '472b6623f57a41568fe7ffce8e40e350'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.199394 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '472b6623f57a41568fe7ffce8e40e350'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.200188 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '472b6623f57a41568fe7ffce8e40e350'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.208391 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '472b6623f57a41568fe7ffce8e40e350'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.221760 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '472b6623f57a41568fe7ffce8e40e350'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.228250 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1454/2913] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:39:19 2022] DELETE /identity/v3/users/472b6623f57a41568fe7ffce8e40e350 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:39:19.406666 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:19.416985 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8de34c-be22-4e5a-b058-3ccb786cdd58 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:19.486609 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:19.487241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:19.487515 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:19.487814 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] PATH_INFO: `/v3/projects/09397aeb58f44f7c9f0ab921e471079a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:19.488909 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09397aeb58f44f7c9f0ab921e471079a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:19.490784 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:19.539320 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09397aeb58f44f7c9f0ab921e471079a', 'request_id': 'req-363e6783-c3f4-4997-aac0-a2074e9b0782'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.550311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09397aeb58f44f7c9f0ab921e471079a', 'request_id': 'req-363e6783-c3f4-4997-aac0-a2074e9b0782'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.560062 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09397aeb58f44f7c9f0ab921e471079a', 'request_id': 'req-363e6783-c3f4-4997-aac0-a2074e9b0782'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.571991 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] The token cache is being invalidate because project 09397aeb58f44f7c9f0ab921e471079a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:39:19.572277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.573140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.574804 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1460/2914] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:39:19 2022] DELETE /identity/v3/projects/09397aeb58f44f7c9f0ab921e471079a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:39:19.578563 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:19.580487 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:19.594650 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa77977-804f-4860-a4ed-29b7bf4ba615 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:19.650962 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:19.652160 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:19.652441 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:19.652901 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] PATH_INFO: `/v3/users/a773d2f22532427b9ca87d721c7798a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:19.654040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a773d2f22532427b9ca87d721c7798a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:19.656001 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:19.702448 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a773d2f22532427b9ca87d721c7798a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.709589 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a773d2f22532427b9ca87d721c7798a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.710212 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a773d2f22532427b9ca87d721c7798a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.717720 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a773d2f22532427b9ca87d721c7798a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.725685 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:39:19.730907 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a773d2f22532427b9ca87d721c7798a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:19.737434 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1455/2915] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:39:19 2022] DELETE /identity/v3/users/a773d2f22532427b9ca87d721c7798a0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:39:19.907628 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:19.909048 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:19.922610 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e6783-c3f4-4997-aac0-a2074e9b0782 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:19.944008 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:39:19.982463 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:19.983105 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:19.983386 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:19.983701 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] PATH_INFO: `/v3/projects/81f3ef01e71946bb96dc617898d0d8b8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:19.984820 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f3ef01e71946bb96dc617898d0d8b8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:19.986747 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:20.033862 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f3ef01e71946bb96dc617898d0d8b8', 'request_id': 'req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:20.039303 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f3ef01e71946bb96dc617898d0d8b8', 'request_id': 'req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:20.047055 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f3ef01e71946bb96dc617898d0d8b8', 'request_id': 'req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:20.058789 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] The token cache is being invalidate because project 81f3ef01e71946bb96dc617898d0d8b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:39:20.059073 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:20.059717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:39:20.061165 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1461/2916] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:39:19 2022] DELETE /identity/v3/projects/81f3ef01e71946bb96dc617898d0d8b8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:39:23.367531 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:23.368995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:23.424684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07579f5-95f8-4f9c-8e52-3c5af8bb39c1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:23.475475 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:23.476040 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:23.476319 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:23.476593 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:23.478522 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:23.585368 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:23.587181 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:23.600407 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1456/2917] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:39:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:39:27.242398 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:39:27.243895 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:27.256902 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b7c7c-2e75-4a5c-93f6-e36c520e3f40 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:39:27.260400 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:39:27.261986 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:39:27.262266 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:39:27.262537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:39:27.264202 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:39:27.278100 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:39:27.279525 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:39:27.290508 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1462/2918] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:39:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:39:33.594374 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=77,keystone:DELETE=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:39:37.285600 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:40:10.576538 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:10.578679 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:10.596406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea671b5-3735-4f2f-ac4e-22a684be148d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:10.605923 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:10.606935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:10.607571 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:10.608583 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:10.612157 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:10.628207 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:10.630560 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:10.645058 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1457/2919] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:40:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:40:10.733170 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:10.743904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b53329-3f91-4372-990f-45fb830401da service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:10.750826 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:10.752132 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:10.752702 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:10.753369 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:10.759333 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:10.770185 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:10.771682 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503a38a-bc37-4d9b-9fd7-ce2e6022f174 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:10.781733 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1463/2920] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:40:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:40:11.871467 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:11.884290 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54e63-8f1d-4f15-aaab-c3a48f71b3ec service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:11.890313 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:11.890995 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:11.891334 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:11.891704 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:11.893562 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:11.907284 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:11.909106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:11.924477 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1458/2921] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:40:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:40:18.222937 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:18.223415 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:18.223530 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:18.272585 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] MFA Rules not processed for user `e0acbc7456fe40a1b0d70c0a35976fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 06:40:18.273800 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:18.338311 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1464/2922] 10.222.0.25 () {58 vars in 967 bytes} [Fri Sep 9 06:40:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 06:40:18.378420 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:18.379866 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:18.467630 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e235ae8-9fbc-4e69-b075-e3b3fe2e0020 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:18.472928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb474b7f08db4ff9b9fdf8eb8ca658af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:18.473527 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:18.473804 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:18.474076 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:18.482766 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:18.484400 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:18.492225 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1459/2923] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Sep 9 06:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 06:40:21.712520 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:21.714568 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:21.815365 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c1b15-9ecf-4847-82fb-f9b3b59de0ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:21.822075 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:21.822718 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:21.823000 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:21.826863 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:21.828573 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:21.846755 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:21.848569 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:21.860173 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1465/2924] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:40:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:40:25.173655 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:25.175148 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:25.253822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bbb9-a9f8-4201-9011-b03153c0cdea service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:25.258343 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:25.258967 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:25.259245 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:25.259517 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] PATH_INFO: `/v3/users/baef9538b75b4244b6ccf0922af7e5b1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:25.260699 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=baef9538b75b4244b6ccf0922af7e5b1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:25.262607 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:25.308199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:40:25.309884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baef9538b75b4244b6ccf0922af7e5b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.317283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baef9538b75b4244b6ccf0922af7e5b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.317961 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baef9538b75b4244b6ccf0922af7e5b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.325085 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baef9538b75b4244b6ccf0922af7e5b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.337161 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baef9538b75b4244b6ccf0922af7e5b1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.343320 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1460/2925] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:40:25 2022] DELETE /identity/v3/users/baef9538b75b4244b6ccf0922af7e5b1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:40:25.520513 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:25.532626 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25b63b-eb14-4b03-8816-6d140075d259 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:25.586470 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:40:25.599140 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:25.599880 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:25.600157 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:25.600426 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] PATH_INFO: `/v3/projects/3312523776514be3a56e169c9234168b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:25.601537 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3312523776514be3a56e169c9234168b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:25.603431 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:25.651338 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3312523776514be3a56e169c9234168b', 'request_id': 'req-d7a7a40e-a995-4653-8ddd-a510fc58fca5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.656872 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3312523776514be3a56e169c9234168b', 'request_id': 'req-d7a7a40e-a995-4653-8ddd-a510fc58fca5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.664199 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3312523776514be3a56e169c9234168b', 'request_id': 'req-d7a7a40e-a995-4653-8ddd-a510fc58fca5'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.676277 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] The token cache is being invalidate because project 3312523776514be3a56e169c9234168b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:40:25.676595 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.677235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.678786 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1466/2926] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:40:25 2022] DELETE /identity/v3/projects/3312523776514be3a56e169c9234168b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:40:25.683443 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:25.684892 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:25.697883 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c44f8f-9d08-45eb-9d80-987e71bbd30d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:25.753501 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:25.754125 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:25.754401 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:25.754674 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] PATH_INFO: `/v3/users/200e1ff9d8c946b288d1e4b69640ee64` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:25.755822 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=200e1ff9d8c946b288d1e4b69640ee64)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:25.757725 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:25.802513 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200e1ff9d8c946b288d1e4b69640ee64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.810354 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200e1ff9d8c946b288d1e4b69640ee64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.810977 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200e1ff9d8c946b288d1e4b69640ee64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.818062 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200e1ff9d8c946b288d1e4b69640ee64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.830549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200e1ff9d8c946b288d1e4b69640ee64'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:25.836347 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1461/2927] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:40:25 2022] DELETE /identity/v3/users/200e1ff9d8c946b288d1e4b69640ee64 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:40:26.009643 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:40:26.011067 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:40:26.023717 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7a40e-a995-4653-8ddd-a510fc58fca5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:40:26.083215 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:40:26.083873 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:40:26.084150 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:40:26.084417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] PATH_INFO: `/v3/projects/26d791d43e96429dbba858b56dbf73f2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:40:26.085507 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d791d43e96429dbba858b56dbf73f2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:40:26.087384 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:40:26.133728 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d791d43e96429dbba858b56dbf73f2', 'request_id': 'req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:26.139860 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d791d43e96429dbba858b56dbf73f2', 'request_id': 'req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:26.147656 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d791d43e96429dbba858b56dbf73f2', 'request_id': 'req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:26.159534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] The token cache is being invalidate because project 26d791d43e96429dbba858b56dbf73f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:40:26.159833 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:26.160449 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:40:26.161903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1467/2928] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:40:26 2022] DELETE /identity/v3/projects/26d791d43e96429dbba858b56dbf73f2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:40:35.832994 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=66,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:40:36.156602 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:41:17.467283 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:41:17.468770 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.533711 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6419f-603a-4971-8297-311fbe9979ab admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:41:17.586298 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:41:17.586935 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:41:17.587209 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:41:17.587481 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:41:17.589291 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.599988 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:41:17.602015 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:41:17.610191 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1462/2929] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 06:41:17.682425 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:41:17.683836 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.696436 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120cc3-aaa9-429c-bb9d-e79fb6e8a017 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:41:17.699977 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86cd9a25b7ed4206a9b5939024168475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:41:17.700574 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:41:17.700851 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:41:17.701116 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:41:17.702769 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.776468 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:41:17.777904 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:41:17.789773 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1468/2930] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:41:17.828559 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:41:17.830549 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.850642 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfa720-f5b5-49ba-9559-452486f6029e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:41:17.855724 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc56896d10094aed91db84ea2bf9dfb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:41:17.856330 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:41:17.856663 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:41:17.856918 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:41:17.857059 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:41:17.858418 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.858734 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.873578 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:41:17.875251 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:41:17.887888 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1463/2931] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:41:17.934430 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9e5dd-1bc4-4e6a-8206-9cd9903c477f service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:41:17.939629 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:41:17.940236 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:41:17.940518 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:41:17.940790 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:41:17.942546 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:41:17.954112 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:41:17.955719 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:41:17.965793 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1469/2932] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:41:27.883344 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=52 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:41:27.961371 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=66 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:42:52.507316 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:42:52.518631 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ace73-4d7b-4742-8c15-5bf623c5567e service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:42:52.524166 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:42:52.524768 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:42:52.525049 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:42:52.525365 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:42:52.527199 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:42:52.539579 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:42:52.541184 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:42:52.552828 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1464/2933] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:42:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:42:56.555861 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:42:56.557580 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:42:56.571762 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1ed6a-dba0-4edf-ab13-c00c9f1cb2af service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:42:56.577235 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31e2bef8256c40c68bc7711c5922289d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63076cf4a77649a2bdc642cd4f18db4e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:42:56.577919 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:42:56.578391 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:42:56.578734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:42:56.581548 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:42:56.593607 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:42:56.595988 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:42:56.607870 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1470/2934] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Sep 9 06:42:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 06:42:59.763787 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:42:59.765240 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:42:59.830877 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939f039-f961-432a-8d39-5fd82fae569b service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:42:59.835457 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:42:59.836124 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:42:59.836406 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:42:59.836678 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] PATH_INFO: `/v3/users/b84d34323691437b9e2e9e99c29db0f1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:42:59.837815 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84d34323691437b9e2e9e99c29db0f1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:42:59.839757 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:42:59.885599 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84d34323691437b9e2e9e99c29db0f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:42:59.893269 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84d34323691437b9e2e9e99c29db0f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:42:59.893940 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84d34323691437b9e2e9e99c29db0f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:42:59.900928 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84d34323691437b9e2e9e99c29db0f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:42:59.912987 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84d34323691437b9e2e9e99c29db0f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:42:59.918767 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1465/2935] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:42:59 2022] DELETE /identity/v3/users/b84d34323691437b9e2e9e99c29db0f1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:43:00.086589 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:43:00.099135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0258c57-7140-4eaf-ac4b-aafdaeec6f60 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:43:00.161903 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:43:00.162534 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:43:00.162812 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:43:00.163086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] PATH_INFO: `/v3/projects/240d1fbee4bc4dbeb359f463302be8ee` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:43:00.164210 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240d1fbee4bc4dbeb359f463302be8ee)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:43:00.166111 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:43:00.214882 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240d1fbee4bc4dbeb359f463302be8ee', 'request_id': 'req-142c36c5-684b-4b64-9ccc-eb1c824fc967'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.220771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240d1fbee4bc4dbeb359f463302be8ee', 'request_id': 'req-142c36c5-684b-4b64-9ccc-eb1c824fc967'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.227837 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240d1fbee4bc4dbeb359f463302be8ee', 'request_id': 'req-142c36c5-684b-4b64-9ccc-eb1c824fc967'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.240129 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] The token cache is being invalidate because project 240d1fbee4bc4dbeb359f463302be8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:43:00.240417 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.241128 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.242615 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1471/2936] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:43:00 2022] DELETE /identity/v3/projects/240d1fbee4bc4dbeb359f463302be8ee => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 06:43:00.246684 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:43:00.248167 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:43:00.261842 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37f744-355c-4ad1-84e7-f0ff7f048f20 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:43:00.319061 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:43:00.319729 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:43:00.320006 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:43:00.320274 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] PATH_INFO: `/v3/users/5c1db29862c04c4d812ecb467d058992` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:43:00.321497 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1db29862c04c4d812ecb467d058992)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:43:00.323681 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:43:00.330106 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:43:00.370577 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1db29862c04c4d812ecb467d058992'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.377884 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1db29862c04c4d812ecb467d058992'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.378490 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1db29862c04c4d812ecb467d058992'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.385282 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1db29862c04c4d812ecb467d058992'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.397439 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4b36288-dee4-426d-85d9-bc1ab1fb574e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1db29862c04c4d812ecb467d058992'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.403159 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1466/2937] 10.222.0.25 () {58 vars in 1224 bytes} [Fri Sep 9 06:43:00 2022] DELETE /identity/v3/users/5c1db29862c04c4d812ecb467d058992 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:43:00.581713 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 06:43:00.583135 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 06:43:00.597261 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c36c5-684b-4b64-9ccc-eb1c824fc967 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 06:43:00.655117 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '285daab328104f75886dde4ae3928fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df82cd1e5393473c92978bf22c792e3e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 06:43:00.655771 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 06:43:00.656047 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 06:43:00.656316 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] PATH_INFO: `/v3/projects/e2a72949ee104e32b5cf777e7fe3de8e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 06:43:00.657506 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2a72949ee104e32b5cf777e7fe3de8e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 06:43:00.659397 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 06:43:00.679748 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:43:00.706192 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2a72949ee104e32b5cf777e7fe3de8e', 'request_id': 'req-4a519a29-ac4a-45b6-837d-055a6b2b81a4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.711968 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2a72949ee104e32b5cf777e7fe3de8e', 'request_id': 'req-4a519a29-ac4a-45b6-837d-055a6b2b81a4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.719734 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2a72949ee104e32b5cf777e7fe3de8e', 'request_id': 'req-4a519a29-ac4a-45b6-837d-055a6b2b81a4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.731086 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] The token cache is being invalidate because project e2a72949ee104e32b5cf777e7fe3de8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 06:43:00.731370 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.732029 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4a519a29-ac4a-45b6-837d-055a6b2b81a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 06:43:00.733486 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1472/2938] 10.222.0.25 () {58 vars in 1233 bytes} [Fri Sep 9 06:43:00 2022] DELETE /identity/v3/projects/e2a72949ee104e32b5cf777e7fe3de8e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 06:43:10.399816 dv-cinder-852209-16-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 06:43:10.728241 dv-cinder-852209-16-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}